Repository: massalabs/DeWeb
Pull request #264: Improve caching and speed
Author: login: damip
Created: 2025-03-23T22:19:16Z | Merged: 2025-03-24T12:31:32Z

* removed the zip caching stuff
* added a 2 level cache (hot on LRU in RAM, cold on disk using Badger)
* added a website file list cache as well to avoid querying that list for every file path 

Overall performance is significantly better


Diff:
diff --git a/.github/actions/install/action.yml b/.github/actions/install/action.yml
index e276ddee..195126e3 100644
--- a/.github/actions/install/action.yml
+++ b/.github/actions/install/action.yml
@@ -14,7 +14,7 @@ runs:
   steps:
     - uses: actions/setup-go@v5
       with:
-        go-version: 1.22
+        go-version: 1.23
         cache: true
         cache-dependency-path: server/go.sum
 
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
index 93d97129..3734bab9 100644
--- a/.github/workflows/lint.yml
+++ b/.github/workflows/lint.yml
@@ -25,7 +25,7 @@ jobs:
       - name: run golangci-lint
         uses: golangci/golangci-lint-action@v6
         with:
-          version: v1.59
+          version: v1.64
           args: --timeout=3m
           working-directory: ./server
 
diff --git a/server/api/read/restapi/configure_de_web.go b/server/api/read/restapi/configure_de_web.go
index fc73af58..d1853917 100644
--- a/server/api/read/restapi/configure_de_web.go
+++ b/server/api/read/restapi/configure_de_web.go
@@ -10,6 +10,7 @@ import (
 	"github.com/go-openapi/runtime"
 	"github.com/go-openapi/runtime/middleware"
 	"github.com/massalabs/deweb-server/api/read/restapi/operations"
+	"github.com/massalabs/deweb-server/pkg/webmanager"
 	"github.com/massalabs/station/pkg/logger"
 )
 
@@ -43,7 +44,13 @@ func configureAPI(api *operations.DeWebAPI) http.Handler {
 		})
 	}
 
-	api.PreServerShutdown = func() {}
+	api.PreServerShutdown = func() {
+		if webmanager.CacheInstance() != nil {
+			if err := webmanager.CacheInstance().Close(); err != nil {
+				logger.Errorf("Failed to close cache: %v", err)
+			}
+		}
+	}
 
 	api.ServerShutdown = func() {}
 
diff --git a/server/go.mod b/server/go.mod
index ab921b63..dcaeefea 100644
--- a/server/go.mod
+++ b/server/go.mod
@@ -1,15 +1,18 @@
 module github.com/massalabs/deweb-server
 
-go 1.22
+go 1.23
+
+toolchain go1.23.2
 
 require (
+	github.com/dgraph-io/badger/v3 v3.2103.5
 	github.com/go-openapi/errors v0.22.0
 	github.com/go-openapi/loads v0.22.0
 	github.com/go-openapi/runtime v0.28.0
 	github.com/go-openapi/spec v0.21.0
 	github.com/go-openapi/strfmt v0.23.0
 	github.com/go-openapi/swag v0.23.0
-	github.com/gofrs/flock v0.12.1
+	github.com/hashicorp/golang-lru v1.0.2
 	github.com/jessevdk/go-flags v1.6.1
 	github.com/massalabs/station v0.6.5
 	github.com/massalabs/station-massa-wallet v0.4.5
@@ -22,28 +25,41 @@ require (
 	github.com/awnumar/memcall v0.1.2 // indirect
 	github.com/awnumar/memguard v0.22.3 // indirect
 	github.com/btcsuite/btcutil v1.0.2 // indirect
+	github.com/cespare/xxhash v1.1.0 // indirect
+	github.com/cespare/xxhash/v2 v2.1.1 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/dgraph-io/ristretto v0.1.1 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
+	github.com/dustin/go-humanize v1.0.0 // indirect
 	github.com/go-openapi/analysis v0.23.0 // indirect
 	github.com/go-openapi/jsonpointer v0.21.0 // indirect
 	github.com/go-openapi/jsonreference v0.21.0 // indirect
 	github.com/go-openapi/validate v0.24.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // indirect
+	github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6 // indirect
+	github.com/golang/protobuf v1.3.1 // indirect
+	github.com/golang/snappy v0.0.3 // indirect
+	github.com/google/flatbuffers v1.12.1 // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/josharian/intern v1.0.0 // indirect
+	github.com/klauspost/compress v1.13.6 // indirect
 	github.com/klauspost/cpuid/v2 v2.1.0 // indirect
 	github.com/mailru/easyjson v0.7.7 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
 	github.com/oklog/ulid v1.3.1 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/shopspring/decimal v1.3.1 // indirect
-	github.com/stretchr/testify v1.9.0 // indirect
+	github.com/stretchr/testify v1.10.0 // indirect
 	github.com/ybbus/jsonrpc/v3 v3.1.4 // indirect
 	go.mongodb.org/mongo-driver v1.14.0 // indirect
+	go.opencensus.io v0.22.5 // indirect
 	go.uber.org/atomic v1.10.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.uber.org/zap v1.24.0 // indirect
 	golang.org/x/crypto v0.32.0 // indirect
-	golang.org/x/sync v0.6.0 // indirect
+	golang.org/x/sync v0.10.0 // indirect
 	golang.org/x/sys v0.29.0 // indirect
 	gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
diff --git a/server/go.sum b/server/go.sum
index cb8979b9..c4841e2a 100644
--- a/server/go.sum
+++ b/server/go.sum
@@ -1,4 +1,9 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=
+github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
 github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII=
+github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
 github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
 github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
 github.com/awnumar/memcall v0.1.2 h1:7gOfDTL+BJ6nnbtAp9+HQzUFjtP1hEseRQq8eP055QY=
@@ -17,11 +22,29 @@ github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVa
 github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc=
 github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY=
 github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs=
+github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko=
+github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
+github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY=
+github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
+github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
+github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
+github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
 github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dgraph-io/badger/v3 v3.2103.5 h1:ylPa6qzbjYRQMU6jokoj4wzcaweHylt//CH0AKt0akg=
+github.com/dgraph-io/badger/v3 v3.2103.5/go.mod h1:4MPiseMeDQ3FNCYwRbbcBOGJLf5jsE0PPFzRiKjtcdw=
+github.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWajOK8=
+github.com/dgraph-io/ristretto v0.1.1/go.mod h1:S1GPSBCYCIhmVNfcth17y2zZtQT6wzkzgwUve0VDWWA=
+github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2 h1:tdlZCpZ/P9DhczCTSixgIKmwPv6+wP5DGjqLYw5SUiA=
+github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
 github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
 github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
+github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
+github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
 github.com/go-openapi/analysis v0.23.0 h1:aGday7OWupfMs+LbmLZG4k0MYXIANxcuBTYUC03zFCU=
 github.com/go-openapi/analysis v0.23.0/go.mod h1:9mz9ZWaSlV8TvjQHLl2mUW2PbZtemkE8yA5v22ohupo=
@@ -43,14 +66,31 @@ github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+Gr
 github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
 github.com/go-openapi/validate v0.24.0 h1:LdfDKwNbpB6Vn40xhTdNZAnfLECL81w+VX3BumrGD58=
 github.com/go-openapi/validate v0.24.0/go.mod h1:iyeX1sEufmv3nPbBdX3ieNviWnOZaJ1+zquzJEf2BAQ=
-github.com/gofrs/flock v0.12.1 h1:MTLVXXHf8ekldpJk3AKicLij9MdwOWkZ+a/jHHZby9E=
-github.com/gofrs/flock v0.12.1/go.mod h1:9zxTsyu5xtJ9DK+1tFZyibEV7y3uwDxPPfbxeeHCoD0=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6 h1:ZgQEtGgCBiWRM39fZuwSd1LwSqqSW0hOdXCYYDX0R3I=
+github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
 github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
+github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA=
+github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
+github.com/google/flatbuffers v1.12.1 h1:MVlul7pQNoDzWRLTw5imwYsl+usrS1TXG2H4jg6ImGw=
+github.com/google/flatbuffers v1.12.1/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
+github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
 github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=
+github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
+github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
 github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
+github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
 github.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
 github.com/jessevdk/go-flags v1.6.1 h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bBK4=
@@ -58,20 +98,31 @@ github.com/jessevdk/go-flags v1.6.1/go.mod h1:Mk8T1hIAWpOiJiHa9rJASDK2UGWji0EuPG
 github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
 github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=
+github.com/klauspost/compress v1.12.3/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
+github.com/klauspost/compress v1.13.6 h1:P76CopJELS0TiO2mebmnzgWaajssP/EszplttgQxcgc=
+github.com/klauspost/compress v1.13.6/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
 github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
 github.com/klauspost/cpuid/v2 v2.1.0 h1:eyi1Ad2aNJMW95zcSbmGg7Cg6cq3ADwLpMAP96d8rF0=
 github.com/klauspost/cpuid/v2 v2.1.0/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
+github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
 github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/massalabs/station v0.6.5 h1:lkWWYB0/dPsRHSoAqN0wh/mFx8+InsVGMfraXQFZ76U=
 github.com/massalabs/station v0.6.5/go.mod h1:i9d/zAqeDYxY0Od5LEfEkP6V2osUhA13GwA72ByM8bQ=
 github.com/massalabs/station-massa-wallet v0.4.5 h1:0rTHxGPlJ5cKjgB/yQclOBHbWiZO5rOwO0lT7ZjFuVQ=
 github.com/massalabs/station-massa-wallet v0.4.5/go.mod h1:Eu6Zlijs0uAuGM5CxEUOxFrcIlWtuZVAbiWPCUni9XY=
+github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
+github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
 github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
 github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
 github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
@@ -79,22 +130,42 @@ github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn
 github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
 github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
 github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
+github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M=
 github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA=
+github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
 github.com/shopspring/decimal v1.3.1 h1:2Usl1nmF/WZucqkFZhnfFYxxxu8LG21F6nPQBE5gKV8=
 github.com/shopspring/decimal v1.3.1/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
+github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
+github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
+github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
+github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
+github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
+github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
+github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
+github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
+github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
 github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
-github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
-github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
+github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
+github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
+github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
 github.com/ybbus/jsonrpc/v3 v3.1.4 h1:pPmgfWXnqR2GdIlealyCzmV6LV3nxm3w9gwA1B3cP3Y=
 github.com/ybbus/jsonrpc/v3 v3.1.4/go.mod h1:4HQTl0UzErqWGa6bSXhp8rIjifMAMa55E4D5wdhe768=
+github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 go.mongodb.org/mongo-driver v1.14.0 h1:P98w8egYRjYe3XDjxhYJagTokP/H6HzlsnojRgZRd80=
 go.mongodb.org/mongo-driver v1.14.0/go.mod h1:Vzb0Mk/pa7e6cWw85R4F/endUC3u0U9jGcNU603k65c=
+go.opencensus.io v0.22.5 h1:dntmOdLpSpHlVqbW5Eay97DelsZHe+55D+xC6i0dDS0=
+go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
 go.uber.org/atomic v1.10.0 h1:9qC72Qh0+3MqyJbAn8YU5xVq1frD8bn3JtD2oXtafVQ=
 go.uber.org/atomic v1.10.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
 go.uber.org/goleak v1.1.11 h1:wy28qYRKZgnJTxGxvye5/wgWr1EKjmUDGYox5mGlRlI=
@@ -104,26 +175,72 @@ go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN8
 go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60=
 go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg=
 golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20200115085410-6d4e4cb37c7d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc=
 golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc=
+golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
+golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.34.0 h1:Mb7Mrk043xzHgnRM88suvJFwzVrRfHEHJEl5/71CKw0=
 golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
-golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
+golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
 golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
+golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
 gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
@@ -132,10 +249,12 @@ gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYs
 gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
 gopkg.in/yaml.v1 v1.0.0-20140924161607-9f9df34309c0/go.mod h1:WDnlLJ4WF5VGsH/HVa3CI79GS0ol3YnhVnKP89i0kNg=
 gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
 gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 howett.net/plist v1.0.0 h1:7CrbWYbPPO/PyNy38b2EB/+gYbjCe2DXBxgtOOZbSQM=
 howett.net/plist v1.0.0/go.mod h1:lqaXoTrLY4hg8tnEzNru53gicrbv7rrk+2xJA/7hw9g=
 lukechampine.com/blake3 v1.1.7 h1:GgRMhmdsuK8+ii6UZFDL8Nb+VyMwadAgcJyfYHxG6n0=
diff --git a/server/int/api/config/config.go b/server/int/api/config/config.go
index b47676f6..7a331112 100644
--- a/server/int/api/config/config.go
+++ b/server/int/api/config/config.go
@@ -14,33 +14,47 @@ const (
 	DefaultDomain         = "localhost"
 	DefaultNetworkNodeURL = "https://mainnet.massa.net/api/v2"
 	DefaultAPIPort        = 8080
+
+	// Default cache size limits
+	DefaultMaxRAMEntries                uint64 = 1000  // Maximum number of file entries in RAM cache
+	DefaultMaxDiskEntries               uint64 = 10000 // Maximum number of file entries in disk cache
+	DefaultFileListCacheDurationSeconds        = 60    // Default expiration of the file list cache
 )
 
 type ServerConfig struct {
-	Domain       string
-	APIPort      int
-	NetworkInfos msConfig.NetworkInfos
-	AllowList    []string
-	BlockList    []string
+	Domain                       string
+	APIPort                      int
+	NetworkInfos                 msConfig.NetworkInfos
+	AllowList                    []string
+	BlockList                    []string
+	SiteRAMCacheMaxItems         uint64
+	SiteDiskCacheMaxItems        uint64
+	FileListCacheDurationSeconds int
 }
 
 type yamlServerConfig struct {
-	Domain         string   `yaml:"domain"`
-	NetworkNodeURL string   `yaml:"network_node_url"`
-	APIPort        int      `yaml:"api_port"`
-	AllowList      []string `yaml:"allow_list"`
-	BlockList      []string `yaml:"block_list"`
+	Domain                       string   `yaml:"domain"`
+	NetworkNodeURL               string   `yaml:"network_node_url"`
+	APIPort                      int      `yaml:"api_port"`
+	AllowList                    []string `yaml:"allow_list"`
+	BlockList                    []string `yaml:"block_list"`
+	SiteRAMCacheMaxItems         uint64   `yaml:"site_ram_cache_max_items"`
+	SiteDiskCacheMaxItems        uint64   `yaml:"site_disk_cache_max_items"`
+	FileListCacheDurationSeconds int      `yaml:"file_list_cache_duration_seconds"`
 }
 
 func DefaultConfig() *ServerConfig {
 	networkInfos := pkgConfig.NewNetworkConfig(DefaultNetworkNodeURL)
 
 	return &ServerConfig{
-		Domain:       DefaultDomain,
-		APIPort:      DefaultAPIPort,
-		NetworkInfos: networkInfos,
-		AllowList:    []string{},
-		BlockList:    []string{},
+		Domain:                       DefaultDomain,
+		APIPort:                      DefaultAPIPort,
+		NetworkInfos:                 networkInfos,
+		AllowList:                    []string{},
+		BlockList:                    []string{},
+		SiteRAMCacheMaxItems:         DefaultMaxRAMEntries,
+		SiteDiskCacheMaxItems:        DefaultMaxDiskEntries,
+		FileListCacheDurationSeconds: DefaultFileListCacheDurationSeconds,
 	}
 }
 
@@ -79,13 +93,29 @@ func LoadServerConfig(configPath string) (*ServerConfig, error) {
 		yamlConf.APIPort = DefaultAPIPort
 	}
 
+	// Set default values for cache settings if not specified
+	if yamlConf.SiteRAMCacheMaxItems == 0 {
+		yamlConf.SiteRAMCacheMaxItems = DefaultMaxRAMEntries
+	}
+
+	if yamlConf.SiteDiskCacheMaxItems == 0 {
+		yamlConf.SiteDiskCacheMaxItems = DefaultMaxDiskEntries
+	}
+
+	if yamlConf.FileListCacheDurationSeconds == 0 {
+		yamlConf.FileListCacheDurationSeconds = DefaultFileListCacheDurationSeconds
+	}
+
 	networkInfos := pkgConfig.NewNetworkConfig(yamlConf.NetworkNodeURL)
 
 	return &ServerConfig{
-		Domain:       yamlConf.Domain,
-		APIPort:      yamlConf.APIPort,
-		NetworkInfos: networkInfos,
-		AllowList:    yamlConf.AllowList,
-		BlockList:    yamlConf.BlockList,
+		Domain:                       yamlConf.Domain,
+		APIPort:                      yamlConf.APIPort,
+		NetworkInfos:                 networkInfos,
+		AllowList:                    yamlConf.AllowList,
+		BlockList:                    yamlConf.BlockList,
+		SiteRAMCacheMaxItems:         yamlConf.SiteRAMCacheMaxItems,
+		SiteDiskCacheMaxItems:        yamlConf.SiteDiskCacheMaxItems,
+		FileListCacheDurationSeconds: yamlConf.FileListCacheDurationSeconds,
 	}, nil
 }
diff --git a/server/pages/src/components/RootApp.tsx b/server/pages/src/components/RootApp.tsx
new file mode 100644
index 00000000..c1becfd7
--- /dev/null
+++ b/server/pages/src/components/RootApp.tsx
@@ -0,0 +1,15 @@
+// @ts-expect-error - React is used for JSX
+import React, { ReactNode, StrictMode } from "react";
+
+interface RootAppProps {
+  children: ReactNode;
+  theme?: string;
+}
+
+export function RootApp({ children }: RootAppProps) {
+  return (
+    <StrictMode>
+      {children}
+    </StrictMode>
+  );
+} 
\ No newline at end of file
diff --git a/server/pages/src/utils/RootApp.tsx b/server/pages/src/utils/RootApp.tsx
index 43f889be..c0abc404 100644
--- a/server/pages/src/utils/RootApp.tsx
+++ b/server/pages/src/utils/RootApp.tsx
@@ -1,5 +1,7 @@
-import { ReactNode, StrictMode } from "react";
+// @ts-expect-error - React is used for JSX
+import React, { ReactNode } from "react";
 import { createRoot } from "react-dom/client";
+import { RootApp } from "../components/RootApp";
 
 import "../index.css";
 
@@ -9,15 +11,6 @@ interface RootAppProps {
   theme?: string;
 }
 
-
-function RootApp({ children }: RootAppProps) {
-  return (
-    <StrictMode>
-      {children}
-    </StrictMode>
-  );
-}
-
 export function renderApp({ children }: RootAppProps) {
   createRoot(document.getElementById("root")!).render(
     <RootApp>{children}</RootApp>
diff --git a/server/pkg/cache/cache.go b/server/pkg/cache/cache.go
index 9ca1ce81..e872a9d8 100644
--- a/server/pkg/cache/cache.go
+++ b/server/pkg/cache/cache.go
@@ -1,303 +1,215 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
 	"fmt"
-	"io"
-	"os"
-	"path/filepath"
+	"hash/fnv"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
 )
 
-// Cache represents the caching system
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache      *lru.Cache
+	diskCache     *DiskCache
+	mu            sync.RWMutex
+	maxRAMEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
-
-	return &Cache{CacheDir: cacheDir}, nil
-}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Initialize disk cache
+		diskCache, err := NewDiskCache(cacheDir, maxDiskEntries)
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize disk cache: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			diskCache:     diskCache,
+			maxRAMEntries: maxRAMEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
 
-	return flock.New(lockFilePath)
-}
+			// Save evicted entry to disk
+			_ = instance.diskCache.SaveResource(entry)
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+	})
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	return instance, initErr
+}
 
-	defer lockUnlock(fileLock)
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	return h.Sum64()
+}
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+// GetLastModified returns the last modified time of a resource in the cache for a given website
+func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer closeReader(r)
+	key := getHashKey(websiteAddress, fileName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
 		}
 	}
 
-	return false
-}
-
-// GetLastModified returns the last modified time of a resource in the cache for a given website
-func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
+	// If not in RAM cache, try disk cache without promoting
+	modified, err := c.diskCache.GetLastModified(websiteAddress, fileName)
 	if err != nil {
-		//nolint:wrapcheck
 		return time.Time{}, err
 	}
 
-	defer lockUnlock(fileLock)
+	return modified, nil
+}
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
-	}
+// Read returns the content of a resource in the cache for a given website
+func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer closeReader(r)
+	key := getHashKey(websiteAddress, resourceName)
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
 		}
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
-}
-
-// Read returns the content of a resource in the cache for a given website
-func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
+	// If not in RAM cache, check disk cache and move to RAM cache
+	content, modified, err := c.diskCache.RemoveAndGet(websiteAddress, resourceName)
 	if err != nil {
-		//nolint:wrapcheck
 		return nil, err
 	}
 
-	defer lockUnlock(fileLock)
-
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       modified,
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
 	}
 
-	defer closeReader(r)
-
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
-
-			defer closeReader(rc)
-
-			buf := new(bytes.Buffer)
-
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
-
-			return buf.Bytes(), nil
-		}
-	}
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
+	return content, nil
 }
 
 // Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
-
-	err := writeLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return err
-	}
-
-	defer lockUnlock(fileLock)
+func (c *Cache) Save(websiteAddress string, resourceName string, content []byte, modified time.Time) error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
-	if err != nil {
-		return err
+	key := getHashKey(websiteAddress, resourceName)
+	entry := &cacheEntry{
+		content:        content,
+		modified:       modified,
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
-
-	wf, err := w.CreateHeader(header)
-	if err != nil {
-		return fmt.Errorf("failed to create header for new file %s: %v", resourceName, err)
-	}
-
-	_, err = wf.Write(content)
-	if err != nil {
-		return fmt.Errorf("failed to write content for new file %s: %v", resourceName, err)
-	}
-
-	err = w.Close()
-	if err != nil {
-		return fmt.Errorf("failed to close zip writer: %v", err)
-	}
+	// Remove any existing entry from RAM cache
+	c.ramCache.Remove(key)
 
-	err = os.WriteFile(tempArchivePath, buf.Bytes(), os.ModePerm)
-	if err != nil {
-		return fmt.Errorf("failed to write zip archive to temp file %s: %v", tempArchivePath, err)
+	// Remove any existing entry from disk cache
+	if err := c.diskCache.Remove(websiteAddress, resourceName); err != nil {
+		return fmt.Errorf("failed to remove existing entry from disk cache: %v", err)
 	}
 
-	err = os.Rename(tempArchivePath, archivePath)
-	if err != nil {
-		os.Remove(tempArchivePath)
-		return fmt.Errorf("failed to replace old zip archive %s with new one %s: %v", archivePath, tempArchivePath, err)
-	}
+	// Add the new entry to RAM cache
+	c.ramCache.Add(key, entry)
 
 	return nil
 }
 
 // Delete a resource in the cache for a given website
 func (c *Cache) Delete(websiteAddress string, resourceName string) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
-
-	err := writeLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
 
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
+	// Remove from RAM cache
+	c.ramCache.Remove(key)
 
-	err = c.copyExistingEntries(archivePath, w, resourceName)
-	if err != nil {
-		return err
-	}
-
-	err = w.Close()
-	if err != nil {
-		return fmt.Errorf("failed to close zip writer: %v", err)
-	}
-
-	err = os.WriteFile(tempArchivePath, buf.Bytes(), os.ModePerm)
-	if err != nil {
-		return fmt.Errorf("failed to write zip archive to temp file %s: %v", tempArchivePath, err)
-	}
-
-	err = os.Rename(tempArchivePath, archivePath)
-	if err != nil {
-		os.Remove(tempArchivePath)
-		return fmt.Errorf("failed to replace old zip archive %s with new one %s: %v", archivePath, tempArchivePath, err)
+	// Remove from disk cache
+	if err := c.diskCache.Remove(websiteAddress, resourceName); err != nil {
+		return fmt.Errorf("failed to delete from disk cache: %v", err)
 	}
 
 	return nil
 }
 
-// copyExistingEntries copies existing ZIP entries to a new ZIP writer, excluding the specified resource
-func (c *Cache) copyExistingEntries(archivePath string, w *zip.Writer, excludeResource string) error {
-	_, err := os.Stat(archivePath)
-	if err != nil && os.IsNotExist(err) {
-		return nil
-	}
-
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return fmt.Errorf("failed to open existing zip archive %s: %v", archivePath, err)
-	}
-
-	defer closeReader(r)
-
-	for _, f := range r.File {
-		if f.Name == excludeResource {
+// Close saves all RAM cache entries to disk and closes the database
+func (c *Cache) Close() error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	// Get all keys from RAM cache in LRU order
+	keys := c.ramCache.Keys()
+	for _, key := range keys {
+		// Get the entry from RAM cache
+		value, ok := c.ramCache.Peek(key.(uint64))
+		if !ok {
 			continue
 		}
 
-		err = copyZipEntry(f, w)
-		if err != nil {
-			return fmt.Errorf("failed to copy file %s inside zip %s: %v", f.Name, archivePath, err)
+		entry, ok := value.(*cacheEntry)
+		if !ok {
+			continue
 		}
-	}
 
-	return nil
-}
-
-// copyZipEntry copies a zip entry from an existing zip to a new zip writer
-func copyZipEntry(f *zip.File, w *zip.Writer) error {
-	rc, err := f.Open()
-	if err != nil {
-		return fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-	}
-
-	defer closeReader(rc)
-
-	wrc, err := w.CreateHeader(&f.FileHeader)
-	if err != nil {
-		return fmt.Errorf("failed to create header for file %s: %v", f.Name, err)
+		// Save to disk cache
+		if err := c.diskCache.SaveResource(entry); err != nil {
+			return fmt.Errorf("failed to save entry to disk cache: %v", err)
+		}
 	}
 
-	_, err = io.Copy(wrc, rc)
-	if err != nil {
-		return fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
+	// Close the disk cache
+	if err := c.diskCache.Close(); err != nil {
+		return fmt.Errorf("failed to close disk cache: %v", err)
 	}
 
 	return nil
 }
 
-// closeReader closes a reader and logs any errors
-func closeReader(r io.Closer) {
-	err := r.Close()
-	if err != nil {
-		logger.Errorf("failed to close reader: %v", err)
-	}
+// cacheEntry represents a cached resource with its content and modification time
+type cacheEntry struct {
+	content        []byte
+	modified       time.Time
+	websiteAddress string
+	resourceName   string
 }
diff --git a/server/pkg/cache/cache_test.go b/server/pkg/cache/cache_test.go
new file mode 100644
index 00000000..19c8b41a
--- /dev/null
+++ b/server/pkg/cache/cache_test.go
@@ -0,0 +1,52 @@
+package cache
+
+import (
+	"bytes"
+	"fmt"
+	"testing"
+	"time"
+)
+
+func TestCacheItems(t *testing.T) {
+	// Use t.TempDir() which is designed for test isolation
+	tmpDir := t.TempDir()
+
+	// Initialize cache
+	cache, err := NewCache(tmpDir, 10, 1000)
+	if err != nil {
+		t.Fatalf("Failed to create cache: %v", err)
+	}
+	defer cache.Close()
+
+	// Test data
+	website := "test-website.com"
+	numItems := 100
+	items := make(map[string][]byte)
+
+	// Save items to the cache
+	for i := 0; i < numItems; i++ {
+		fileName := fmt.Sprintf("file%d.txt", i)
+		content := []byte(fmt.Sprintf("Content for file %d", i))
+		items[fileName] = content
+
+		err = cache.Save(website, fileName, content, time.Now())
+		if err != nil {
+			t.Fatalf("Failed to save item %d: %v", i, err)
+		}
+	}
+
+	// Read all items back and verify their content
+	for i := 0; i < numItems; i++ {
+		fileName := fmt.Sprintf("file%d.txt", i)
+		expectedContent := items[fileName]
+
+		content, err := cache.Read(website, fileName)
+		if err != nil {
+			t.Fatalf("Failed to read item %d: %v", i, err)
+		}
+
+		if !bytes.Equal(content, expectedContent) {
+			t.Errorf("Content mismatch for item %d:\nExpected: %s\nGot: %s", i, expectedContent, content)
+		}
+	}
+}
diff --git a/server/pkg/cache/disk_cache.go b/server/pkg/cache/disk_cache.go
new file mode 100644
index 00000000..3959690a
--- /dev/null
+++ b/server/pkg/cache/disk_cache.go
@@ -0,0 +1,437 @@
+package cache
+
+import (
+	"encoding/binary"
+	"fmt"
+	"os"
+	"time"
+
+	"github.com/dgraph-io/badger/v3"
+)
+
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+)
+
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
+
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
+
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
+
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
+
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
+
+	// Write resource name
+	copy(buf[offset:], resourceName)
+
+	return buf
+}
+
+// createEntryPrefix returns a prefix key for a website and resource entry
+// Note: We create a new byte slice and copy data rather than using append
+// because Badger requires variables within a transaction to have stable
+// underlying storage. Modifying slices in-place can cause bugs with Badger
+// as it may reference the memory later.
+func createEntryPrefix(websiteAddress, resourceName string) []byte {
+	cacheKey := getCacheKey(websiteAddress, resourceName)
+	prefix := make([]byte, 1+len(cacheKey))
+	prefix[0] = entryTag
+	copy(prefix[1:], cacheKey)
+
+	return prefix
+}
+
+// createIdKey returns a key for an entry's ID counter
+// Note: We create a new byte slice and copy data rather than using append
+// because Badger requires variables within a transaction to have stable
+// underlying storage. Modifying slices in-place can cause bugs with Badger
+// as it may reference the memory later.
+func createIdKey(entryPrefix []byte) []byte {
+	key := make([]byte, len(entryPrefix)+1)
+	copy(key, entryPrefix)
+	key[len(entryPrefix)] = entrySubTagID
+
+	return key
+}
+
+// createDataKey returns a key for an entry's content data
+// Note: We create a new byte slice and copy data rather than using append
+// because Badger requires variables within a transaction to have stable
+// underlying storage. Modifying slices in-place can cause bugs with Badger
+// as it may reference the memory later.
+func createDataKey(entryPrefix []byte) []byte {
+	key := make([]byte, len(entryPrefix)+1)
+	copy(key, entryPrefix)
+	key[len(entryPrefix)] = entrySubTagData
+
+	return key
+}
+
+// createTimestampKey returns a key for an entry's timestamp
+// Note: We create a new byte slice and copy data rather than using append
+// because Badger requires variables within a transaction to have stable
+// underlying storage. Modifying slices in-place can cause bugs with Badger
+// as it may reference the memory later.
+func createTimestampKey(entryPrefix []byte) []byte {
+	key := make([]byte, len(entryPrefix)+1)
+	copy(key, entryPrefix)
+	key[len(entryPrefix)] = entrySubTagTime
+
+	return key
+}
+
+// createIdCounterIndexKey returns a key for an entry in the ID counter index
+// Note: We create a new byte slice and copy data rather than using append
+// because Badger requires variables within a transaction to have stable
+// underlying storage. Modifying slices in-place can cause bugs with Badger
+// as it may reference the memory later.
+func createIdCounterIndexKey(id uint64) []byte {
+	key := make([]byte, 1+8)
+	key[0] = idCounterIndexTag
+	binary.BigEndian.PutUint64(key[1:], id)
+
+	return key
+}
+
+// DiskCache represents the disk storage component of the cache
+type DiskCache struct {
+	db         *badger.DB
+	idCounter  uint64
+	entryCount uint64
+	maxEntries uint64
+}
+
+// NewDiskCache initializes the disk cache with configurable maximum size
+func NewDiskCache(cacheDir string, maxEntries uint64) (*DiskCache, error) {
+	// Check if cache directory exists and is writable
+	if err := os.MkdirAll(cacheDir, 0o755); err != nil {
+		return nil, fmt.Errorf("failed to create cache directory: %v", err)
+	}
+
+	// Initialize BadgerDB
+	opts := badger.DefaultOptions(cacheDir)
+	opts.Logger = nil // Disable BadgerDB's logger
+
+	db, err := badger.Open(opts)
+	if err != nil {
+		return nil, fmt.Errorf("failed to open BadgerDB: %v", err)
+	}
+
+	// Initialize disk cache instance
+	diskCache := &DiskCache{
+		db:         db,
+		maxEntries: maxEntries,
+	}
+
+	// Initialize or load the ID counter and count entries
+	err = db.Update(func(txn *badger.Txn) error {
+		// Find the highest existing ID counter value and count entries
+		it := txn.NewIterator(badger.DefaultIteratorOptions)
+		defer it.Close()
+
+		// Initialize counters
+		diskCache.idCounter = 0
+		diskCache.entryCount = 0
+
+		// Count entries by iterating through all ID counter index entries
+		for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+			key := it.Item().Key()
+			if len(key) > 1 {
+				id := binary.BigEndian.Uint64(key[1:])
+				if id+1 > diskCache.idCounter {
+					diskCache.idCounter = id + 1
+				}
+
+				diskCache.entryCount++
+			}
+		}
+
+		return nil
+	})
+	if err != nil {
+		db.Close()
+		return nil, fmt.Errorf("failed to initialize database: %v", err)
+	}
+
+	return diskCache, nil
+}
+
+// getTimestamp retrieves and parses a timestamp from the database
+func (d *DiskCache) getTimestamp(txn *badger.Txn, entryPrefix []byte) (time.Time, error) {
+	// Create timestamp key
+	timestampKey := createTimestampKey(entryPrefix)
+
+	// Get the timestamp item
+	item, err := txn.Get(timestampKey)
+	if err == badger.ErrKeyNotFound {
+		return time.Time{}, fmt.Errorf("timestamp not found")
+	}
+
+	if err != nil {
+		return time.Time{}, err
+	}
+
+	// Read and parse timestamp value
+	timestampValue, err := item.ValueCopy(nil)
+	if err != nil {
+		return time.Time{}, err
+	}
+
+	// Convert binary to time.Time
+	return time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue))), nil
+}
+
+// GetLastModified returns the last modified time of a resource in the disk cache
+func (d *DiskCache) GetLastModified(websiteAddress, resourceName string) (time.Time, error) {
+	var modified time.Time
+
+	err := d.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryPrefix := createEntryPrefix(websiteAddress, resourceName)
+
+		// Get the timestamp
+		timestamp, err := d.getTimestamp(txn, entryPrefix)
+		if err != nil {
+			return fmt.Errorf("timestamp not found for website %s, resource %s: %v", websiteAddress, resourceName, err)
+		}
+
+		modified = timestamp
+
+		return nil
+	})
+	if err != nil {
+		return time.Time{}, err
+	}
+
+	return modified, nil
+}
+
+// deleteEntry removes an entry from the disk cache
+func (d *DiskCache) deleteEntry(txn *badger.Txn, websiteAddress, resourceName string) error {
+	// Create the entry key prefix
+	entryPrefix := createEntryPrefix(websiteAddress, resourceName)
+
+	// Get the ID counter for this entry
+	idCounterKey := createIdKey(entryPrefix)
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
+
+	if err != nil {
+		return err
+	}
+
+	idCounterValue, err := item.ValueCopy(nil)
+	if err != nil {
+		return err
+	}
+
+	// Create data and timestamp keys
+	dataKey := createDataKey(entryPrefix)
+	timestampKey := createTimestampKey(entryPrefix)
+
+	// Delete the ID counter entry
+	if err := txn.Delete(idCounterKey); err != nil {
+		return fmt.Errorf("failed to delete ID counter entry: %v", err)
+	}
+
+	// Delete the data entry
+	if err := txn.Delete(dataKey); err != nil {
+		return fmt.Errorf("failed to delete data entry: %v", err)
+	}
+
+	// Delete the timestamp entry
+	if err := txn.Delete(timestampKey); err != nil {
+		return fmt.Errorf("failed to delete timestamp entry: %v", err)
+	}
+
+	// Delete the ID counter index entry
+	idCounterIndexKey := createIdCounterIndexKey(binary.BigEndian.Uint64(idCounterValue))
+	if err := txn.Delete(idCounterIndexKey); err != nil {
+		return fmt.Errorf("failed to delete ID counter index: %v", err)
+	}
+
+	// Update entry count in memory only
+	if d.entryCount > 0 {
+		d.entryCount--
+	}
+
+	return nil
+}
+
+// saveEntry saves an entry to the disk cache
+func (d *DiskCache) saveEntry(txn *badger.Txn, websiteAddress, resourceName string, content []byte, modified time.Time) error {
+	// Create the entry key prefix
+	entryPrefix := createEntryPrefix(websiteAddress, resourceName)
+
+	// Create keys
+	idCounterKey := createIdKey(entryPrefix)
+	dataKey := createDataKey(entryPrefix)
+	timestampKey := createTimestampKey(entryPrefix)
+
+	// Save the ID counter
+	idCounterValue := make([]byte, 8)
+	binary.BigEndian.PutUint64(idCounterValue, d.idCounter)
+
+	if err := txn.Set(idCounterKey, idCounterValue); err != nil {
+		return fmt.Errorf("failed to save ID counter: %v", err)
+	}
+
+	// Save the data
+	if err := txn.Set(dataKey, content); err != nil {
+		return fmt.Errorf("failed to save data: %v", err)
+	}
+
+	// Save the timestamp
+	timestampValue := make([]byte, 8)
+	binary.BigEndian.PutUint64(timestampValue, uint64(modified.UnixNano()))
+
+	if err := txn.Set(timestampKey, timestampValue); err != nil {
+		return fmt.Errorf("failed to save timestamp: %v", err)
+	}
+
+	// Save the ID counter index
+	idCounterIndexKey := createIdCounterIndexKey(d.idCounter)
+
+	// Create cache key for the index value
+	indexValue := getCacheKey(websiteAddress, resourceName)
+	if err := txn.Set(idCounterIndexKey, indexValue); err != nil {
+		return fmt.Errorf("failed to save ID counter index: %v", err)
+	}
+
+	// Update entry count in memory only
+	d.entryCount++
+
+	// Increment ID counter
+	d.idCounter++
+
+	return nil
+}
+
+// evictOldestEntry removes the oldest entry from the disk cache
+func (d *DiskCache) evictOldestEntry(txn *badger.Txn) error {
+	it := txn.NewIterator(badger.DefaultIteratorOptions)
+	defer it.Close()
+
+	// Find the oldest entry by looking at the lowest ID counter index
+	seekKey := []byte{idCounterIndexTag}
+	it.Seek(seekKey)
+
+	if !it.ValidForPrefix(seekKey) {
+		return nil // No entries to evict
+	}
+
+	// Get the key to delete
+	cacheKey, err := it.Item().ValueCopy(nil)
+	if err != nil {
+		return err
+	}
+
+	// Extract website and resource name from cache key
+	websiteLen := binary.BigEndian.Uint64(cacheKey[:8])
+	websiteAddress := string(cacheKey[8 : 8+websiteLen])
+
+	resourceOffset := 8 + websiteLen
+	resourceLen := binary.BigEndian.Uint64(cacheKey[resourceOffset : resourceOffset+8])
+	resourceName := string(cacheKey[resourceOffset+8 : resourceOffset+8+resourceLen])
+
+	// Delete the entry
+	return d.deleteEntry(txn, websiteAddress, resourceName)
+}
+
+// SaveResource saves a resource to the disk cache, evicting old entries if necessary
+func (d *DiskCache) SaveResource(entry *cacheEntry) error {
+	return d.db.Update(func(txn *badger.Txn) error {
+		// Make sure we have space by evicting old entries if needed
+		for d.entryCount >= d.maxEntries {
+			if err := d.evictOldestEntry(txn); err != nil {
+				return err
+			}
+		}
+
+		// Delete existing entry if exists
+		if err := d.deleteEntry(txn, entry.websiteAddress, entry.resourceName); err != nil {
+			return err
+		}
+
+		// Save the new entry
+		return d.saveEntry(txn, entry.websiteAddress, entry.resourceName, entry.content, entry.modified)
+	})
+}
+
+// Remove removes a resource from the disk cache
+func (d *DiskCache) Remove(websiteAddress, resourceName string) error {
+	return d.db.Update(func(txn *badger.Txn) error {
+		return d.deleteEntry(txn, websiteAddress, resourceName)
+	})
+}
+
+// Close closes the database
+func (d *DiskCache) Close() error {
+	return d.db.Close()
+}
+
+// RemoveAndGet retrieves a resource from disk cache, removes it, and returns its content and timestamp
+func (d *DiskCache) RemoveAndGet(websiteAddress, resourceName string) ([]byte, time.Time, error) {
+	var content []byte
+	var modified time.Time
+
+	err := d.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryPrefix := createEntryPrefix(websiteAddress, resourceName)
+
+		// Create data key
+		dataKey := createDataKey(entryPrefix)
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("data not found for website %s, resource %s", websiteAddress, resourceName)
+		}
+
+		if err != nil {
+			return err
+		}
+
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
+
+		// Get the timestamp
+		timestamp, err := d.getTimestamp(txn, entryPrefix)
+		if err != nil {
+			return fmt.Errorf("timestamp not found for website %s, resource %s: %v", websiteAddress, resourceName, err)
+		}
+		modified = timestamp
+
+		// Remove from disk
+		return d.deleteEntry(txn, websiteAddress, resourceName)
+	})
+	if err != nil {
+		return nil, time.Time{}, err
+	}
+
+	return content, modified, nil
+}
diff --git a/server/pkg/cache/flock.go b/server/pkg/cache/flock.go
deleted file mode 100644
index 425b40c1..00000000
--- a/server/pkg/cache/flock.go
+++ /dev/null
@@ -1,52 +0,0 @@
-package cache
-
-import (
-	"context"
-	"fmt"
-	"time"
-
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
-)
-
-// writeLockFile tries to acquire a write lock on a file
-func writeLockFile(fileLock *flock.Flock) error {
-	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
-	defer cancel()
-
-	locked, err := fileLock.TryLockContext(ctx, 50*time.Millisecond)
-	if err != nil {
-		return fmt.Errorf("failed to acquire file lock on %s: %v", fileLock.Path(), err)
-	}
-
-	if !locked {
-		return fmt.Errorf("timed out trying to acquire file lock on %s", fileLock.Path())
-	}
-
-	return nil
-}
-
-// readLockFile tries to acquire a read lock on a file
-func readLockFile(fileLock *flock.Flock) error {
-	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
-	defer cancel()
-
-	locked, err := fileLock.TryRLockContext(ctx, 50*time.Millisecond)
-	if err != nil {
-		return fmt.Errorf("failed to acquire file lock on %s: %v", fileLock.Path(), err)
-	}
-
-	if !locked {
-		return fmt.Errorf("timed out trying to acquire file lock on %s", fileLock.Path())
-	}
-
-	return nil
-}
-
-// lockUnlock unlocks a file lock
-func lockUnlock(fileLock *flock.Flock) {
-	err := fileLock.Unlock()
-	if err != nil {
-		logger.Errorf("failed to unlock file %s: %v", fileLock.Path(), err)
-	}
-}
diff --git a/server/pkg/webmanager/manager.go b/server/pkg/webmanager/manager.go
index f2e762a8..1bbce505 100644
--- a/server/pkg/webmanager/manager.go
+++ b/server/pkg/webmanager/manager.go
@@ -3,6 +3,7 @@ package webmanager
 import (
 	"fmt"
 
+	"github.com/massalabs/deweb-server/int/api/config"
 	"github.com/massalabs/deweb-server/pkg/cache"
 	"github.com/massalabs/deweb-server/pkg/website"
 	msConfig "github.com/massalabs/station/int/config"
@@ -11,12 +12,53 @@ import (
 
 const cacheDir = "./websitesCache/"
 
+var (
+	cacheInstance *cache.Cache
+	cacheErr      error
+	serverConfig  *config.ServerConfig
+)
+
+// SetConfig sets the server configuration for the webmanager
+func SetConfig(config *config.ServerConfig) {
+	serverConfig = config
+}
+
+// CacheInstance returns the global cache instance
+func CacheInstance() *cache.Cache {
+	return cacheInstance
+}
+
+// initCache initializes the cache instance if it hasn't been initialized yet
+func initCache() error {
+	if cacheInstance == nil {
+		// If serverConfig is not set, log a warning and use default values
+		var maxRAMEntries uint64 = config.DefaultMaxRAMEntries
+		var maxDiskEntries uint64 = config.DefaultMaxDiskEntries
+
+		if serverConfig != nil {
+			maxRAMEntries = serverConfig.SiteRAMCacheMaxItems
+			maxDiskEntries = serverConfig.SiteDiskCacheMaxItems
+		} else {
+			logger.Debugf("Server config not set, using default cache values")
+		}
+
+		cacheInstance, cacheErr = cache.NewCache(cacheDir, maxRAMEntries, maxDiskEntries)
+		if cacheErr != nil {
+			logger.Errorf("Failed to create cache: %v", cacheErr)
+			return cacheErr
+		}
+	}
+
+	return nil
+}
+
 // getWebsiteResource fetches a resource from a website and returns its content.
 func GetWebsiteResource(network *msConfig.NetworkInfos, websiteAddress, resourceName string) ([]byte, error) {
 	logger.Debugf("Getting website %s resource %s", websiteAddress, resourceName)
 
 	content, err := RequestFile(websiteAddress, network, resourceName)
 	if err != nil {
+		logger.Debugf("GetWebsiteResource failed")
 		return nil, fmt.Errorf("failed to get file %s from website %s: %w", resourceName, websiteAddress, err)
 	}
 
@@ -27,82 +69,60 @@ func GetWebsiteResource(network *msConfig.NetworkInfos, websiteAddress, resource
 
 // RequestFile fetches a website and caches it, or retrieves it from the cache if already present.
 func RequestFile(scAddress string, networkInfo *msConfig.NetworkInfos, resourceName string) ([]byte, error) {
-	cache, err := cache.NewCache(cacheDir)
-	if err != nil {
-		logger.Errorf("Failed to create cache: %v", err)
+	// Initialize cache if needed
+	if err := initCache(); err != nil {
+		logger.Errorf("Failed to initialize cache: %v", err)
 	}
 
-	if cache != nil && cache.IsPresent(scAddress, resourceName) {
-		logger.Debugf("Resource %s from %s present in cache", resourceName, scAddress)
-
-		isOutdated, err := isFileOutdated(cache, networkInfo, scAddress, resourceName)
+	// Get the last update timestamp from the website
+	lastUpdated, err := website.GetLastUpdateTimestamp(networkInfo, scAddress)
+	if err != nil {
+		logger.Warnf("Failed to get last update timestamp: %v", err)
+	} else if cacheInstance != nil {
+		lastModified, err := cacheInstance.GetLastModified(scAddress, resourceName)
 		if err != nil {
-			logger.Warnf("Failed to check if file is outdated: %v", err)
-		} else {
-			if !isOutdated {
-				content, err := cache.Read(scAddress, resourceName)
-				if err != nil {
-					logger.Warnf("Failed to read cached resource %s from %s: %v", resourceName, scAddress, err)
-				} else {
-					return content, nil
-				}
+			logger.Debugf("Resource %s from %s not in cache", resourceName, scAddress)
+		} else if !lastModified.Before(*lastUpdated) {
+			content, err := cacheInstance.Read(scAddress, resourceName)
+			if err != nil {
+				logger.Warnf("Failed to read cached resource %s from %s: %v", resourceName, scAddress, err)
 			} else {
-				if err = cache.Delete(scAddress, resourceName); err != nil {
-					logger.Warnf("Failed to delete outdated resource %s from %s: %v", resourceName, scAddress, err)
-				}
+				logger.Debugf("RequestFile completed (cache hit)")
+				return content, nil
+			}
+		} else {
+			if err = cacheInstance.Delete(scAddress, resourceName); err != nil {
+				logger.Warnf("Failed to delete outdated resource %s from %s: %v", resourceName, scAddress, err)
 			}
-		}
 
-		logger.Warnf("website %s is outdated, fetching...", resourceName)
+			logger.Warnf("website %s is outdated, fetching...", resourceName)
+		}
 	}
 
 	logger.Debugf("Website %s not found in cache or not up to date, fetching...", scAddress)
 
-	websiteBytes, err := fetchAndCache(networkInfo, scAddress, cache, resourceName)
-	if err != nil {
-		return nil, fmt.Errorf("failed to fetch and cache website: %w", err)
-	}
-
-	return websiteBytes, nil
-}
-
-// Fetches the website and saves it to the cache.
-func fetchAndCache(networkInfo *msConfig.NetworkInfos, scAddress string, cache *cache.Cache, resourceName string) ([]byte, error) {
+	// Fetch the website content
 	websiteBytes, err := website.Fetch(networkInfo, scAddress, resourceName)
 	if err != nil {
+		logger.Debugf("RequestFile failed")
 		return nil, fmt.Errorf("failed to fetch %s from %s: %w", resourceName, scAddress, err)
 	}
 
 	logger.Debugf("%s: %s successfully fetched with size: %d bytes", scAddress, resourceName, len(websiteBytes))
 
-	if cache != nil {
-		err := cache.Save(scAddress, resourceName, websiteBytes)
+	// Save to cache if available
+	if cacheInstance != nil {
+		err = cacheInstance.Save(scAddress, resourceName, websiteBytes, *lastUpdated)
 		if err != nil {
-			return nil, fmt.Errorf("failed to save %s to %s cache: %w", resourceName, scAddress, err)
+			logger.Warnf("Failed to save %s to %s cache: %v", resourceName, scAddress, err)
+		} else {
+			logger.Debugf("%s: %s successfully written to cache", scAddress, resourceName)
 		}
-
-		logger.Infof("%s: %s successfully written to cache", scAddress, resourceName)
-	}
-
-	return websiteBytes, nil
-}
-
-// Compares the last updated timestamp to the file's last modified timestamp.
-// Returns true if the file is outdated or if an error occurred. Otherwise, returns false.
-//
-//nolint:unused
-func isFileOutdated(cache *cache.Cache, networkInfo *msConfig.NetworkInfos, scAddress string, fileName string) (bool, error) {
-	lastUpdated, err := website.GetLastUpdateTimestamp(networkInfo, scAddress)
-	if err != nil {
-		return true, fmt.Errorf("failed to get last update timestamp: %w", err)
 	}
 
-	lastModified, err := cache.GetLastModified(scAddress, fileName)
-	if err != nil {
-		return true, fmt.Errorf("failed to get file info: %w", err)
-	}
+	logger.Debugf("RequestFile completed")
 
-	return lastModified.Before(*lastUpdated), nil
+	return websiteBytes, nil
 }
 
 func ResourceExistsOnChain(network *msConfig.NetworkInfos, websiteAddress, filePath string) (bool, error) {
@@ -115,14 +135,3 @@ func ResourceExistsOnChain(network *msConfig.NetworkInfos, websiteAddress, fileP
 
 	return isPresent, nil
 }
-
-func ResourceExistsInCache(scAddress string, resourceName string) (bool, error) {
-	logger.Debugf("Checking if resource %s exists in cache for website %s", resourceName, scAddress)
-
-	cache, err := cache.NewCache(cacheDir)
-	if err != nil {
-		return false, fmt.Errorf("failed to create cache: %w", err)
-	}
-
-	return cache.IsPresent(scAddress, resourceName), nil
-}
diff --git a/server/pkg/website/read.go b/server/pkg/website/read.go
index 32d6cc6b..ec083fa9 100644
--- a/server/pkg/website/read.go
+++ b/server/pkg/website/read.go
@@ -5,10 +5,12 @@ import (
 	"crypto/sha256"
 	"fmt"
 	"strconv"
+	"sync"
 	"time"
 
+	"github.com/massalabs/deweb-server/int/api/config"
 	"github.com/massalabs/deweb-server/pkg/website/storagekeys"
-	"github.com/massalabs/station/int/config"
+	msConfig "github.com/massalabs/station/int/config"
 	"github.com/massalabs/station/pkg/convert"
 	"github.com/massalabs/station/pkg/logger"
 	"github.com/massalabs/station/pkg/node"
@@ -20,8 +22,82 @@ const (
 	lastUpdateTimestampKey = "LAST_UPDATE"
 )
 
+// filePathListCacheEntry represents a cached file path list with its expiration time
+type filePathListCacheEntry struct {
+	files      map[string]struct{} // Using map for O(1) lookups
+	expiration time.Time
+}
+
+// filePathListCache is a thread-safe cache for file path lists
+type filePathListCache struct {
+	mu    sync.RWMutex
+	cache map[string]*filePathListCacheEntry
+}
+
+var (
+	globalFilePathListCache = &filePathListCache{
+		cache: make(map[string]*filePathListCacheEntry),
+	}
+	serverConfig *config.ServerConfig
+)
+
+// SetConfig sets the server configuration for the website package
+func SetConfig(config *config.ServerConfig) {
+	serverConfig = config
+}
+
+// get retrieves the file path list from cache if it exists and is not expired
+func (c *filePathListCache) get(websiteAddress string) (map[string]struct{}, bool) {
+	c.mu.RLock()
+	defer c.mu.RUnlock()
+
+	entry, exists := c.cache[websiteAddress]
+	if !exists {
+		return nil, false
+	}
+
+	if time.Now().After(entry.expiration) {
+		return nil, false
+	}
+
+	return entry.files, true
+}
+
+// set stores the file path list in the cache with an expiration time based on config
+func (c *filePathListCache) set(websiteAddress string, files []string) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	// Clean up expired entries before adding new ones
+	now := time.Now()
+	for key, entry := range c.cache {
+		if now.After(entry.expiration) {
+			delete(c.cache, key)
+		}
+	}
+
+	// Convert slice to map for O(1) lookups
+	filesMap := make(map[string]struct{}, len(files))
+	for _, file := range files {
+		filesMap[file] = struct{}{}
+	}
+
+	// Get expiration duration from config or use default
+	var cacheDuration time.Duration
+	if serverConfig != nil {
+		cacheDuration = time.Duration(serverConfig.FileListCacheDurationSeconds) * time.Second
+	} else {
+		cacheDuration = time.Duration(config.DefaultFileListCacheDurationSeconds) * time.Second
+	}
+
+	c.cache[websiteAddress] = &filePathListCacheEntry{
+		files:      filesMap,
+		expiration: now.Add(cacheDuration),
+	}
+}
+
 // Fetch retrieves the complete data of a website as bytes.
-func Fetch(network *config.NetworkInfos, websiteAddress string, filePath string) ([]byte, error) {
+func Fetch(network *msConfig.NetworkInfos, websiteAddress string, filePath string) ([]byte, error) {
 	client := node.NewClient(network.NodeURL)
 
 	isPresent, err := FilePathExists(network, websiteAddress, filePath)
@@ -82,6 +158,17 @@ func GetFilesPathList(
 	client *node.Client,
 	websiteAddress string,
 ) ([]string, error) {
+	// Try to get from cache first
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		// Convert map back to slice
+		result := make([]string, 0, len(files))
+		for file := range files {
+			result = append(result, file)
+		}
+
+		return result, nil
+	}
+
 	filteredKeys, err := getFileLocationKeys(client, websiteAddress)
 	if err != nil {
 		return nil, fmt.Errorf("fetching website file location keys: %w", err)
@@ -98,6 +185,9 @@ func GetFilesPathList(
 		filesPathList[i] = string(entry.FinalValue)
 	}
 
+	// Store in cache
+	globalFilePathListCache.set(websiteAddress, filesPathList)
+
 	return filesPathList, nil
 }
 
@@ -132,8 +222,11 @@ func fetchAllChunks(client *node.Client, websiteAddress string, filePath string,
 	}
 
 	var dataStore []byte
+	totalBatches := (int(chunkNumber) + datastoreBatchSize - 1) / datastoreBatchSize
+
+	for batch := 0; batch < totalBatches; batch++ {
+		start := batch * datastoreBatchSize
 
-	for start := 0; start < int(chunkNumber); start += datastoreBatchSize {
 		end := start + datastoreBatchSize
 		if end > int(chunkNumber) {
 			end = int(chunkNumber)
@@ -157,13 +250,15 @@ func fetchAllChunks(client *node.Client, websiteAddress string, filePath string,
 
 			dataStore = append(dataStore, entry.FinalValue...)
 		}
+
+		logger.Debugf("Processed batch %d/%d", batch+1, totalBatches)
 	}
 
 	return dataStore, nil
 }
 
 // GetOwner retrieves the owner of the website.
-func GetOwner(network *config.NetworkInfos, websiteAddress string) (string, error) {
+func GetOwner(network *msConfig.NetworkInfos, websiteAddress string) (string, error) {
 	client := node.NewClient(network.NodeURL)
 
 	ownerResponse, err := node.FetchDatastoreEntry(client, websiteAddress, convert.ToBytes(ownerKey))
@@ -175,7 +270,7 @@ func GetOwner(network *config.NetworkInfos, websiteAddress string) (string, erro
 }
 
 // GetLastUpdateTimestamp retrieves the last update timestamp of the website.
-func GetLastUpdateTimestamp(network *config.NetworkInfos, websiteAddress string) (*time.Time, error) {
+func GetLastUpdateTimestamp(network *msConfig.NetworkInfos, websiteAddress string) (*time.Time, error) {
 	client := node.NewClient(network.NodeURL)
 
 	lastUpdateTimestampResponse, err := node.FetchDatastoreEntry(client, websiteAddress, storagekeys.GlobalMetadataKey(convert.ToBytes(lastUpdateTimestampKey)))
@@ -200,7 +295,14 @@ func GetLastUpdateTimestamp(network *config.NetworkInfos, websiteAddress string)
 }
 
 // Check if the requested filePath exists in the SC FilesPathList
-func FilePathExists(network *config.NetworkInfos, websiteAddress string, filePath string) (bool, error) {
+func FilePathExists(network *msConfig.NetworkInfos, websiteAddress string, filePath string) (bool, error) {
+	// Try to get from cache first
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		_, exists := files[filePath]
+		return exists, nil
+	}
+
+	// If not in cache, fetch from chain
 	client := node.NewClient(network.NodeURL)
 	if client == nil {
 		return false, fmt.Errorf("failed to create node client")
@@ -211,10 +313,13 @@ func FilePathExists(network *config.NetworkInfos, websiteAddress string, filePat
 		return false, fmt.Errorf("failed to get files path list: %w", err)
 	}
 
-	for _, file := range files {
-		if file == filePath {
-			return true, nil
-		}
+	// Store in cache for future use
+	globalFilePathListCache.set(websiteAddress, files)
+
+	// Return the result from cache
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		_, exists := files[filePath]
+		return exists, nil
 	}
 
 	return false, nil


Reviews:
user: login: peterjah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T09:05:22Z
user: login: peterjah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T09:06:11Z
user: login: peterjah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T09:11:15Z
user: login: peterjah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T09:14:59Z
user: login: peterjah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T09:29:09Z
user: login: peterjah
body: Well done!
I think the code could be a bit more maintainable by splitting code (especially code of ram cache and badger), factorizing some reused logic and wrapping it into well named functions 
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T09:32:19Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:35:26Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:35:38Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:36:49Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:37:02Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:38:17Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:38:56Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:43:53Z
user: login: peterjah
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T10:59:24Z
user: login: damip
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T11:11:10Z
user: login: peterjah
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T11:18:03Z

Review comments:
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
path: server/pkg/cache/cache.go
user: login: peterjah
body: it seems you can factorize some code with `GetLastModified` function
created_at: 2025-03-24T09:05:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
path: server/pkg/cache/cache.go
user: login: peterjah
body: initialization of diskKey could be done after the ram cache check
created_at: 2025-03-24T09:06:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slices for each key to avoid array modification issues
+		dataKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(dataKey, entryKeyPrefix)
+		dataKey[len(entryKeyPrefix)] = entrySubTagData
+
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("file %s not found in cache for website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			defer closeReader(rc)
+		// Get the timestamp
+		item, err = txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-			buf := new(bytes.Buffer)
+		timestamp, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
+		// Remove from disk
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
 
-			return buf.Bytes(), nil
-		}
+	if err != nil {
+		return nil, err
 	}
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
-}
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       time.Unix(0, int64(binary.BigEndian.Uint64(timestamp))),
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-// Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	err := writeLockFile(fileLock)
+	return content, nil
+}
+
+// deleteEntryFromDisk removes an entry from the disk cache
+func (c *Cache) deleteEntryFromDisk(txn *badger.Txn, key []byte) error {
+	// Create the entry key prefix
+	entryKeyPrefix := make([]byte, 1+len(key))
+	entryKeyPrefix[0] = entryTag
+	copy(entryKeyPrefix[1:], key)
+
+	// Get the ID counter for this entry
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
 	if err != nil {
-		//nolint:wrapcheck
 		return err
 	}
 
-	defer lockUnlock(fileLock)
-
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
+	idCounterValue, err := item.ValueCopy(nil)
 	if err != nil {
 		return err
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
+	// Create data and timestamp keys
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
path: server/pkg/cache/cache.go
user: login: peterjah
body: you can replace these 3 lines by:
`dataKey := append(entryKeyPrefix, entrySubTagData)`

It should be true for all other similar code in your PR
created_at: 2025-03-24T09:11:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slices for each key to avoid array modification issues
+		dataKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(dataKey, entryKeyPrefix)
+		dataKey[len(entryKeyPrefix)] = entrySubTagData
+
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("file %s not found in cache for website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			defer closeReader(rc)
+		// Get the timestamp
+		item, err = txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-			buf := new(bytes.Buffer)
+		timestamp, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
+		// Remove from disk
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
 
-			return buf.Bytes(), nil
-		}
+	if err != nil {
+		return nil, err
 	}
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
-}
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       time.Unix(0, int64(binary.BigEndian.Uint64(timestamp))),
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-// Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	err := writeLockFile(fileLock)
+	return content, nil
+}
+
+// deleteEntryFromDisk removes an entry from the disk cache
+func (c *Cache) deleteEntryFromDisk(txn *badger.Txn, key []byte) error {
+	// Create the entry key prefix
+	entryKeyPrefix := make([]byte, 1+len(key))
+	entryKeyPrefix[0] = entryTag
+	copy(entryKeyPrefix[1:], key)
+
+	// Get the ID counter for this entry
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
 	if err != nil {
-		//nolint:wrapcheck
 		return err
 	}
 
-	defer lockUnlock(fileLock)
-
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
+	idCounterValue, err := item.ValueCopy(nil)
 	if err != nil {
 		return err
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
+	// Create data and timestamp keys
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
 
-	wf, err := w.CreateHeader(header)
-	if err != nil {
-		return fmt.Errorf("failed to create header for new file %s: %v", resourceName, err)
+	timestampKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(timestampKey, entryKeyPrefix)
+	timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+	// Delete the ID counter entry
+	if err := txn.Delete(idCounterKey); err != nil {
+		return fmt.Errorf("failed to delete ID counter entry: %v", err)
 	}
 
-	_, err = wf.Write(content)
-	if err != nil {
-		return fmt.Errorf("failed to write content for new file %s: %v", resourceName, err)
+	// Delete the data entry
+	if err := txn.Delete(dataKey); err != nil {
+		return fmt.Errorf("failed to delete data entry: %v", err)
 	}
 
-	err = w.Close()
-	if err != nil {
-		return fmt.Errorf("failed to close zip writer: %v", err)
+	// Delete the timestamp entry
+	if err := txn.Delete(timestampKey); err != nil {
+		return fmt.Errorf("failed to delete timestamp entry: %v", err)
 	}
 
-	err = os.WriteFile(tempArchivePath, buf.Bytes(), os.ModePerm)
-	if err != nil {
-		return fmt.Errorf("failed to write zip archive to temp file %s: %v", tempArchivePath, err)
+	// Delete the ID counter index entry
+	idCounterIndexKey := make([]byte, 1+len(idCounterValue))
+	idCounterIndexKey[0] = idCounterIndexTag
+	copy(idCounterIndexKey[1:], idCounterValue)
+	if err := txn.Delete(idCounterIndexKey); err != nil {
+		return fmt.Errorf("failed to delete ID counter index: %v", err)
 	}
 
-	err = os.Rename(tempArchivePath, archivePath)
-	if err != nil {
-		os.Remove(tempArchivePath)
-		return fmt.Errorf("failed to replace old zip archive %s with new one %s: %v", archivePath, tempArchivePath, err)
+	// Update entry count in memory only
+	if c.entryCount > 0 {
+		c.entryCount--
 	}
 
 	return nil
 }
 
-// Delete a resource in the cache for a given website
-func (c *Cache) Delete(websiteAddress string, resourceName string) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+// saveEntryToDisk saves an entry to the disk cache
+func (c *Cache) saveEntryToDisk(txn *badger.Txn, entry *cacheEntry) error {
+	// Compute the key from the entry contents
+	key := getCacheKey(entry.websiteAddress, entry.resourceName)
 
-	err := writeLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return err
-	}
+	// Create the entry key prefix
+	entryKeyPrefix := append([]byte{entryTag}, key...)
 
-	defer lockUnlock(fileLock)
+	// Create new byte slices for each key to avoid array modification issues
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
 
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
 
-	err = c.copyExistingEntries(archivePath, w, resourceName)
-	if err != nil {
-		return err
+	timestampKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(timestampKey, entryKeyPrefix)
+	timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+	// Save the ID counter
+	idCounterValue := make([]byte, 8)
+	binary.BigEndian.PutUint64(idCounterValue, c.idCounter)
+	if err := txn.Set(idCounterKey, idCounterValue); err != nil {
+		return fmt.Errorf("failed to save ID counter: %v", err)
 	}
 
-	err = w.Close()
-	if err != nil {
-		return fmt.Errorf("failed to close zip writer: %v", err)
+	// Save the data
+	if err := txn.Set(dataKey, entry.content); err != nil {
+		return fmt.Errorf("failed to save data: %v", err)
 	}
 
-	err = os.WriteFile(tempArchivePath, buf.Bytes(), os.ModePerm)
-	if err != nil {
-		return fmt.Errorf("failed to write zip archive to temp file %s: %v", tempArchivePath, err)
+	// Save the timestamp
+	timestampValue := make([]byte, 8)
+	binary.BigEndian.PutUint64(timestampValue, uint64(entry.modified.UnixNano()))
+	if err := txn.Set(timestampKey, timestampValue); err != nil {
+		return fmt.Errorf("failed to save timestamp: %v", err)
 	}
 
-	err = os.Rename(tempArchivePath, archivePath)
-	if err != nil {
-		os.Remove(tempArchivePath)
-		return fmt.Errorf("failed to replace old zip archive %s with new one %s: %v", archivePath, tempArchivePath, err)
+	// Save the ID counter index
+	idCounterIndexKey := make([]byte, 1+8)
+	idCounterIndexKey[0] = idCounterIndexTag
+	binary.BigEndian.PutUint64(idCounterIndexKey[1:], c.idCounter)
+	if err := txn.Set(idCounterIndexKey, key); err != nil {
+		return fmt.Errorf("failed to save ID counter index: %v", err)
 	}
 
+	// Update entry count in memory only
+	c.entryCount++
+
+	// Increment ID counter
+	c.idCounter++
+
 	return nil
 }
 
-// copyExistingEntries copies existing ZIP entries to a new ZIP writer, excluding the specified resource
-func (c *Cache) copyExistingEntries(archivePath string, w *zip.Writer, excludeResource string) error {
-	_, err := os.Stat(archivePath)
-	if err != nil && os.IsNotExist(err) {
-		return nil
+// evictOldestEntry removes the oldest entry from the disk cache
+func (c *Cache) evictOldestEntry(txn *badger.Txn) error {
+	it := txn.NewIterator(badger.DefaultIteratorOptions)
+	defer it.Close()
+
+	// Find the oldest entry by looking at the lowest ID counter index
+	seekKey := []byte{idCounterIndexTag}
+	it.Seek(seekKey)
+	if !it.ValidForPrefix(seekKey) {
+		return nil // No entries to evict
 	}
 
-	r, err := zip.OpenReader(archivePath)
+	// Get the key to delete
+	key, err := it.Item().ValueCopy(nil)
 	if err != nil {
-		return fmt.Errorf("failed to open existing zip archive %s: %v", archivePath, err)
+		return err
 	}
 
-	defer closeReader(r)
+	// Delete the entry using the existing function
+	return c.deleteEntryFromDisk(txn, key)
+}
 
-	for _, f := range r.File {
-		if f.Name == excludeResource {
-			continue
-		}
+// Save a resource in the cache for a given website
+func (c *Cache) Save(websiteAddress string, resourceName string, content []byte, modified time.Time) error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
+	entry := &cacheEntry{
+		content:        content,
+		modified:       modified,
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-		err = copyZipEntry(f, w)
-		if err != nil {
-			return fmt.Errorf("failed to copy file %s inside zip %s: %v", f.Name, archivePath, err)
-		}
+	// Remove any existing entry from RAM cache
+	c.ramCache.Remove(key)
+
+	// Remove any existing entry from disk cache
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// print the diskKey
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
+
+	if err != nil {
+		return fmt.Errorf("failed to remove existing entry from disk cache: %v", err)
 	}
 
+	// Add the new entry to RAM cache
+	c.ramCache.Add(key, entry)
+
 	return nil
 }
 
-// copyZipEntry copies a zip entry from an existing zip to a new zip writer
-func copyZipEntry(f *zip.File, w *zip.Writer) error {
-	rc, err := f.Open()
+// Delete a resource in the cache for a given website
+func (c *Cache) Delete(websiteAddress string, resourceName string) error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
+
+	// Remove from RAM cache
+	c.ramCache.Remove(key)
+
+	// Remove from disk cache
+	err := c.db.Update(func(txn *badger.Txn) error {
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
+
 	if err != nil {
-		return fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
+		return fmt.Errorf("failed to delete from disk cache: %v", err)
 	}
 
-	defer closeReader(rc)
+	return nil
+}
 
-	wrc, err := w.CreateHeader(&f.FileHeader)
-	if err != nil {
-		return fmt.Errorf("failed to create header for file %s: %v", f.Name, err)
+// Close saves all RAM cache entries to disk and closes the database
+func (c *Cache) Close() error {
path: server/pkg/cache/cache.go
user: login: peterjah
body: Is it really needed? A data in ram should always been written in disk right?
created_at: 2025-03-24T09:14:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -82,6 +158,16 @@ func GetFilesPathList(
 	client *node.Client,
 	websiteAddress string,
 ) ([]string, error) {
+	// Try to get from cache first
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		// Convert map back to slice
+		result := make([]string, 0, len(files))
+		for file := range files {
+			result = append(result, file)
+		}
+		return result, nil
path: server/pkg/website/read.go
user: login: peterjah
body: you can just to this:

`return maps.Keys(files), nil`
created_at: 2025-03-24T09:29:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
path: server/pkg/cache/cache.go
user: login: damip
body: done
created_at: 2025-03-24T10:35:26Z
author_association: MEMBER
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
path: server/pkg/cache/cache.go
user: login: damip
body: moved stuff around, should be cleaner
created_at: 2025-03-24T10:35:38Z
author_association: MEMBER
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slices for each key to avoid array modification issues
+		dataKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(dataKey, entryKeyPrefix)
+		dataKey[len(entryKeyPrefix)] = entrySubTagData
+
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("file %s not found in cache for website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			defer closeReader(rc)
+		// Get the timestamp
+		item, err = txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-			buf := new(bytes.Buffer)
+		timestamp, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
+		// Remove from disk
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
 
-			return buf.Bytes(), nil
-		}
+	if err != nil {
+		return nil, err
 	}
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
-}
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       time.Unix(0, int64(binary.BigEndian.Uint64(timestamp))),
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-// Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	err := writeLockFile(fileLock)
+	return content, nil
+}
+
+// deleteEntryFromDisk removes an entry from the disk cache
+func (c *Cache) deleteEntryFromDisk(txn *badger.Txn, key []byte) error {
+	// Create the entry key prefix
+	entryKeyPrefix := make([]byte, 1+len(key))
+	entryKeyPrefix[0] = entryTag
+	copy(entryKeyPrefix[1:], key)
+
+	// Get the ID counter for this entry
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
 	if err != nil {
-		//nolint:wrapcheck
 		return err
 	}
 
-	defer lockUnlock(fileLock)
-
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
+	idCounterValue, err := item.ValueCopy(nil)
 	if err != nil {
 		return err
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
+	// Create data and timestamp keys
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
path: server/pkg/cache/cache.go
user: login: damip
body: tried it, caused crashes and memory corruption in Badger transactions as it requires immutable underlying memory for variables within transactions and under some circumstances, go optimizes things a bit by rearranging memory on append()
created_at: 2025-03-24T10:36:49Z
author_association: MEMBER
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slices for each key to avoid array modification issues
+		dataKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(dataKey, entryKeyPrefix)
+		dataKey[len(entryKeyPrefix)] = entrySubTagData
+
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("file %s not found in cache for website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			defer closeReader(rc)
+		// Get the timestamp
+		item, err = txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-			buf := new(bytes.Buffer)
+		timestamp, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
+		// Remove from disk
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
 
-			return buf.Bytes(), nil
-		}
+	if err != nil {
+		return nil, err
 	}
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
-}
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       time.Unix(0, int64(binary.BigEndian.Uint64(timestamp))),
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-// Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	err := writeLockFile(fileLock)
+	return content, nil
+}
+
+// deleteEntryFromDisk removes an entry from the disk cache
+func (c *Cache) deleteEntryFromDisk(txn *badger.Txn, key []byte) error {
+	// Create the entry key prefix
+	entryKeyPrefix := make([]byte, 1+len(key))
+	entryKeyPrefix[0] = entryTag
+	copy(entryKeyPrefix[1:], key)
+
+	// Get the ID counter for this entry
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
 	if err != nil {
-		//nolint:wrapcheck
 		return err
 	}
 
-	defer lockUnlock(fileLock)
-
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
+	idCounterValue, err := item.ValueCopy(nil)
 	if err != nil {
 		return err
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
+	// Create data and timestamp keys
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
path: server/pkg/cache/cache.go
user: login: damip
body: added comments in the key construction functions to clarify that bit
created_at: 2025-03-24T10:37:02Z
author_association: MEMBER
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slices for each key to avoid array modification issues
+		dataKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(dataKey, entryKeyPrefix)
+		dataKey[len(entryKeyPrefix)] = entrySubTagData
+
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("file %s not found in cache for website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			defer closeReader(rc)
+		// Get the timestamp
+		item, err = txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-			buf := new(bytes.Buffer)
+		timestamp, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
+		// Remove from disk
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
 
-			return buf.Bytes(), nil
-		}
+	if err != nil {
+		return nil, err
 	}
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
-}
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       time.Unix(0, int64(binary.BigEndian.Uint64(timestamp))),
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-// Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	err := writeLockFile(fileLock)
+	return content, nil
+}
+
+// deleteEntryFromDisk removes an entry from the disk cache
+func (c *Cache) deleteEntryFromDisk(txn *badger.Txn, key []byte) error {
+	// Create the entry key prefix
+	entryKeyPrefix := make([]byte, 1+len(key))
+	entryKeyPrefix[0] = entryTag
+	copy(entryKeyPrefix[1:], key)
+
+	// Get the ID counter for this entry
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
 	if err != nil {
-		//nolint:wrapcheck
 		return err
 	}
 
-	defer lockUnlock(fileLock)
-
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
+	idCounterValue, err := item.ValueCopy(nil)
 	if err != nil {
 		return err
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
+	// Create data and timestamp keys
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
 
-	wf, err := w.CreateHeader(header)
-	if err != nil {
-		return fmt.Errorf("failed to create header for new file %s: %v", resourceName, err)
+	timestampKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(timestampKey, entryKeyPrefix)
+	timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+	// Delete the ID counter entry
+	if err := txn.Delete(idCounterKey); err != nil {
+		return fmt.Errorf("failed to delete ID counter entry: %v", err)
 	}
 
-	_, err = wf.Write(content)
-	if err != nil {
-		return fmt.Errorf("failed to write content for new file %s: %v", resourceName, err)
+	// Delete the data entry
+	if err := txn.Delete(dataKey); err != nil {
+		return fmt.Errorf("failed to delete data entry: %v", err)
 	}
 
-	err = w.Close()
-	if err != nil {
-		return fmt.Errorf("failed to close zip writer: %v", err)
+	// Delete the timestamp entry
+	if err := txn.Delete(timestampKey); err != nil {
+		return fmt.Errorf("failed to delete timestamp entry: %v", err)
 	}
 
-	err = os.WriteFile(tempArchivePath, buf.Bytes(), os.ModePerm)
-	if err != nil {
-		return fmt.Errorf("failed to write zip archive to temp file %s: %v", tempArchivePath, err)
+	// Delete the ID counter index entry
+	idCounterIndexKey := make([]byte, 1+len(idCounterValue))
+	idCounterIndexKey[0] = idCounterIndexTag
+	copy(idCounterIndexKey[1:], idCounterValue)
+	if err := txn.Delete(idCounterIndexKey); err != nil {
+		return fmt.Errorf("failed to delete ID counter index: %v", err)
 	}
 
-	err = os.Rename(tempArchivePath, archivePath)
-	if err != nil {
-		os.Remove(tempArchivePath)
-		return fmt.Errorf("failed to replace old zip archive %s with new one %s: %v", archivePath, tempArchivePath, err)
+	// Update entry count in memory only
+	if c.entryCount > 0 {
+		c.entryCount--
 	}
 
 	return nil
 }
 
-// Delete a resource in the cache for a given website
-func (c *Cache) Delete(websiteAddress string, resourceName string) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+// saveEntryToDisk saves an entry to the disk cache
+func (c *Cache) saveEntryToDisk(txn *badger.Txn, entry *cacheEntry) error {
+	// Compute the key from the entry contents
+	key := getCacheKey(entry.websiteAddress, entry.resourceName)
 
-	err := writeLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return err
-	}
+	// Create the entry key prefix
+	entryKeyPrefix := append([]byte{entryTag}, key...)
 
-	defer lockUnlock(fileLock)
+	// Create new byte slices for each key to avoid array modification issues
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
 
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
 
-	err = c.copyExistingEntries(archivePath, w, resourceName)
-	if err != nil {
-		return err
+	timestampKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(timestampKey, entryKeyPrefix)
+	timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+	// Save the ID counter
+	idCounterValue := make([]byte, 8)
+	binary.BigEndian.PutUint64(idCounterValue, c.idCounter)
+	if err := txn.Set(idCounterKey, idCounterValue); err != nil {
+		return fmt.Errorf("failed to save ID counter: %v", err)
 	}
 
-	err = w.Close()
-	if err != nil {
-		return fmt.Errorf("failed to close zip writer: %v", err)
+	// Save the data
+	if err := txn.Set(dataKey, entry.content); err != nil {
+		return fmt.Errorf("failed to save data: %v", err)
 	}
 
-	err = os.WriteFile(tempArchivePath, buf.Bytes(), os.ModePerm)
-	if err != nil {
-		return fmt.Errorf("failed to write zip archive to temp file %s: %v", tempArchivePath, err)
+	// Save the timestamp
+	timestampValue := make([]byte, 8)
+	binary.BigEndian.PutUint64(timestampValue, uint64(entry.modified.UnixNano()))
+	if err := txn.Set(timestampKey, timestampValue); err != nil {
+		return fmt.Errorf("failed to save timestamp: %v", err)
 	}
 
-	err = os.Rename(tempArchivePath, archivePath)
-	if err != nil {
-		os.Remove(tempArchivePath)
-		return fmt.Errorf("failed to replace old zip archive %s with new one %s: %v", archivePath, tempArchivePath, err)
+	// Save the ID counter index
+	idCounterIndexKey := make([]byte, 1+8)
+	idCounterIndexKey[0] = idCounterIndexTag
+	binary.BigEndian.PutUint64(idCounterIndexKey[1:], c.idCounter)
+	if err := txn.Set(idCounterIndexKey, key); err != nil {
+		return fmt.Errorf("failed to save ID counter index: %v", err)
 	}
 
+	// Update entry count in memory only
+	c.entryCount++
+
+	// Increment ID counter
+	c.idCounter++
+
 	return nil
 }
 
-// copyExistingEntries copies existing ZIP entries to a new ZIP writer, excluding the specified resource
-func (c *Cache) copyExistingEntries(archivePath string, w *zip.Writer, excludeResource string) error {
-	_, err := os.Stat(archivePath)
-	if err != nil && os.IsNotExist(err) {
-		return nil
+// evictOldestEntry removes the oldest entry from the disk cache
+func (c *Cache) evictOldestEntry(txn *badger.Txn) error {
+	it := txn.NewIterator(badger.DefaultIteratorOptions)
+	defer it.Close()
+
+	// Find the oldest entry by looking at the lowest ID counter index
+	seekKey := []byte{idCounterIndexTag}
+	it.Seek(seekKey)
+	if !it.ValidForPrefix(seekKey) {
+		return nil // No entries to evict
 	}
 
-	r, err := zip.OpenReader(archivePath)
+	// Get the key to delete
+	key, err := it.Item().ValueCopy(nil)
 	if err != nil {
-		return fmt.Errorf("failed to open existing zip archive %s: %v", archivePath, err)
+		return err
 	}
 
-	defer closeReader(r)
+	// Delete the entry using the existing function
+	return c.deleteEntryFromDisk(txn, key)
+}
 
-	for _, f := range r.File {
-		if f.Name == excludeResource {
-			continue
-		}
+// Save a resource in the cache for a given website
+func (c *Cache) Save(websiteAddress string, resourceName string, content []byte, modified time.Time) error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
+	entry := &cacheEntry{
+		content:        content,
+		modified:       modified,
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-		err = copyZipEntry(f, w)
-		if err != nil {
-			return fmt.Errorf("failed to copy file %s inside zip %s: %v", f.Name, archivePath, err)
-		}
+	// Remove any existing entry from RAM cache
+	c.ramCache.Remove(key)
+
+	// Remove any existing entry from disk cache
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// print the diskKey
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
+
+	if err != nil {
+		return fmt.Errorf("failed to remove existing entry from disk cache: %v", err)
 	}
 
+	// Add the new entry to RAM cache
+	c.ramCache.Add(key, entry)
+
 	return nil
 }
 
-// copyZipEntry copies a zip entry from an existing zip to a new zip writer
-func copyZipEntry(f *zip.File, w *zip.Writer) error {
-	rc, err := f.Open()
+// Delete a resource in the cache for a given website
+func (c *Cache) Delete(websiteAddress string, resourceName string) error {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
+
+	// Remove from RAM cache
+	c.ramCache.Remove(key)
+
+	// Remove from disk cache
+	err := c.db.Update(func(txn *badger.Txn) error {
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
+
 	if err != nil {
-		return fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
+		return fmt.Errorf("failed to delete from disk cache: %v", err)
 	}
 
-	defer closeReader(rc)
+	return nil
+}
 
-	wrc, err := w.CreateHeader(&f.FileHeader)
-	if err != nil {
-		return fmt.Errorf("failed to create header for file %s: %v", f.Name, err)
+// Close saves all RAM cache entries to disk and closes the database
+func (c *Cache) Close() error {
path: server/pkg/cache/cache.go
user: login: damip
body: RAM is for the hot data, disk is not used at all.
Cold (older) data that does not fit in RAM is sent to disk.

You can see the disk db as a continuation (the tail) of the in-memory LRU.

When closing, badger needs a clean close() and I also want to dump whatever remains in the LRU to disk for persistence
created_at: 2025-03-24T10:38:17Z
author_association: MEMBER
diff_hunk: @@ -82,6 +158,16 @@ func GetFilesPathList(
 	client *node.Client,
 	websiteAddress string,
 ) ([]string, error) {
+	// Try to get from cache first
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		// Convert map back to slice
+		result := make([]string, 0, len(files))
+		for file := range files {
+			result = append(result, file)
+		}
+		return result, nil
path: server/pkg/website/read.go
user: login: damip
body: since go 1.22 the behavior changed and that does not compile
created_at: 2025-03-24T10:38:56Z
author_association: MEMBER
diff_hunk: @@ -1,303 +1,557 @@
 package cache
 
 import (
-	"archive/zip"
-	"bytes"
+	"encoding/binary"
 	"fmt"
-	"io"
+	"hash/fnv"
 	"os"
-	"path/filepath"
+	"sync"
 	"time"
 
-	"github.com/gofrs/flock"
-	"github.com/massalabs/station/pkg/logger"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/dgraph-io/badger/v3"
 )
 
-// Cache represents the caching system
+const (
+	// DB tags for different types of data
+	entryTag          = 0x01
+	entrySubTagID     = 0x01 // Subtag for entry ID counter
+	entrySubTagData   = 0x02 // Subtag for entry data
+	entrySubTagTime   = 0x03 // Subtag for entry timestamp
+	idCounterIndexTag = 0x02
+
+	// Bucket IDs
+	entryBucketID     = 0x10
+)
+
+var (
+	instance *Cache
+	once     sync.Once
+)
+
+// Cache represents the dual caching system with RAM and disk storage
 type Cache struct {
-	CacheDir string
+	ramCache *lru.Cache
+	db       *badger.DB
+	mu       sync.RWMutex
+	// idCounter is a unique counter for each entry in the disk cache
+	idCounter uint64
+	// entryCount tracks the total number of entries in the disk cache
+	entryCount uint64
+	// maxRAMEntries is the maximum number of entries in RAM cache
+	maxRAMEntries uint64
+	// maxDiskEntries is the maximum number of entries in disk cache
+	maxDiskEntries uint64
 }
 
-// NewCache initializes the cache and ensures the cache directory exists
-func NewCache(cacheDir string) (*Cache, error) {
-	err := os.MkdirAll(cacheDir, os.ModePerm)
-	if err != nil {
-		return nil, fmt.Errorf("failed to create cache directory %s: %v", cacheDir, err)
-	}
+// NewCache initializes the cache with configurable maximum sizes for RAM and disk storage
+func NewCache(cacheDir string, maxRAMEntries, maxDiskEntries uint64) (*Cache, error) {
+	var initErr error
+	once.Do(func() {
+		// Check if cache directory exists and is writable
+		if err := os.MkdirAll(cacheDir, 0755); err != nil {
+			initErr = fmt.Errorf("failed to create cache directory: %v", err)
+			return
+		}
 
-	return &Cache{CacheDir: cacheDir}, nil
-}
+		// Initialize BadgerDB
+		opts := badger.DefaultOptions(cacheDir)
+		opts.Logger = nil // Disable BadgerDB's logger
+		db, err := badger.Open(opts)
+		if err != nil {
+			initErr = fmt.Errorf("failed to open BadgerDB: %v", err)
+			return
+		}
 
-// getArchivePath returns the full path to the website's .zip archive
-func (c *Cache) getArchivePath(websiteAddress string) string {
-	return filepath.Join(c.CacheDir, websiteAddress+".zip")
-}
+		// Initialize cache instance
+		instance = &Cache{
+			db:            db,
+			maxRAMEntries: maxRAMEntries,
+			maxDiskEntries: maxDiskEntries,
+		}
 
-// newFileLock creates a new file lock for a given archive path
-func (c *Cache) newFileLock(archivePath string) *flock.Flock {
-	lockFilePath := archivePath + ".lock"
+		// Initialize RAM cache with uint64 type for FNV hash
+		ramCache, err := lru.NewWithEvict(int(maxRAMEntries), func(key interface{}, value interface{}) {
 
-	return flock.New(lockFilePath)
+			// Handle eviction by saving to disk
+			entry, ok := value.(*cacheEntry)
+			if !ok {
+				return
+			}
+
+			// Save evicted entry to disk as the newest entry
+			_ = db.Update(func(txn *badger.Txn) error {
+				// Check if we need to evict the oldest entry from disk
+				for instance.entryCount >= instance.maxDiskEntries {
+					if err := instance.evictOldestEntry(txn); err != nil {
+						return err
+					}
+				}
+
+				// Use the stored website and resource names to create the disk key
+				return instance.saveEntryToDisk(txn, entry)
+			})
+		})
+		if err != nil {
+			initErr = fmt.Errorf("failed to create LRU cache: %v", err)
+			return
+		}
+
+		// Set the RAM cache after successful initialization
+		instance.ramCache = ramCache
+
+		// Initialize or load the ID counter and count entries
+		err = db.Update(func(txn *badger.Txn) error {
+			// Find the highest existing ID counter value and count entries
+			it := txn.NewIterator(badger.DefaultIteratorOptions)
+			defer it.Close()
+
+			// Initialize counters
+			instance.idCounter = 0
+			instance.entryCount = 0
+
+			// Count entries by iterating through all ID counter index entries
+			for it.Seek([]byte{idCounterIndexTag}); it.ValidForPrefix([]byte{idCounterIndexTag}); it.Next() {
+				key := it.Item().Key()
+				if len(key) > 1 {
+					id := binary.BigEndian.Uint64(key[1:])
+					if id + 1 > instance.idCounter {
+						instance.idCounter = id + 1
+					}
+					instance.entryCount++
+				}
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			initErr = fmt.Errorf("failed to initialize database: %v", err)
+			return
+		}
+	})
+	return instance, initErr
 }
 
-// IsPresent returns true if the requested resource is present in the cache for a given website
-func (c *Cache) IsPresent(websiteAddress string, resourceName string) bool {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
+// getCacheKey returns a unique key for a website and resource as a byte slice
+func getCacheKey(websiteAddress, resourceName string) []byte {
+	// Calculate lengths
+	websiteLen := uint64(len(websiteAddress))
+	resourceLen := uint64(len(resourceName))
 
-	err := readLockFile(fileLock)
-	if err != nil {
-		return false
-	}
+	// Create a buffer with enough capacity for all components
+	buf := make([]byte, 8+websiteLen+8+resourceLen)
+	offset := 0
 
-	defer lockUnlock(fileLock)
+	// Write website length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], websiteLen)
+	offset += 8
 
-	_, err = os.Stat(archivePath)
-	if os.IsNotExist(err) {
-		return false
-	}
+	// Write website address
+	copy(buf[offset:], websiteAddress)
+	offset += int(websiteLen)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		logger.Errorf("failed to open zip archive %s: %v", archivePath, err)
-		return false
-	}
+	// Write resource length (8 bytes)
+	binary.BigEndian.PutUint64(buf[offset:], resourceLen)
+	offset += 8
 
-	defer closeReader(r)
+	// Write resource name
+	copy(buf[offset:], resourceName)
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			return true
-		}
-	}
+	return buf
+}
 
-	return false
+// getHashKey returns a uint64 hash for the given website and resource
+func getHashKey(websiteAddress, resourceName string) uint64 {
+	h := fnv.New64a()
+	h.Write([]byte(websiteAddress))
+	h.Write([]byte(resourceName))
+	return h.Sum64()
 }
 
 // GetLastModified returns the last modified time of a resource in the cache for a given website
 func (c *Cache) GetLastModified(websiteAddress string, fileName string) (time.Time, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return time.Time{}, err
-	}
+	c.mu.RLock()
+	defer c.mu.RUnlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, fileName)
+	diskKey := getCacheKey(websiteAddress, fileName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return time.Time{}, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache without promoting
+	if value, ok := c.ramCache.Peek(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.modified, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, try disk cache without promoting
+	var modified time.Time
+	err := c.db.View(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slice for timestamp key
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		item, err := txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", fileName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == fileName {
-			return f.Modified, nil
+		timestampValue, err := item.ValueCopy(nil)
+		if err != nil {
+			return err
 		}
+
+		modified = time.Unix(0, int64(binary.BigEndian.Uint64(timestampValue)))
+		return nil
+	})
+
+	if err != nil {
+		return time.Time{}, err
 	}
 
-	return time.Time{}, fmt.Errorf("file %s not found in archive %s", fileName, archivePath)
+	return modified, nil
 }
 
 // Read returns the content of a resource in the cache for a given website
 func (c *Cache) Read(websiteAddress string, resourceName string) ([]byte, error) {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-
-	err := readLockFile(fileLock)
-	if err != nil {
-		//nolint:wrapcheck
-		return nil, err
-	}
+	c.mu.Lock()
+	defer c.mu.Unlock()
 
-	defer lockUnlock(fileLock)
+	key := getHashKey(websiteAddress, resourceName)
+	diskKey := getCacheKey(websiteAddress, resourceName)
 
-	r, err := zip.OpenReader(archivePath)
-	if err != nil {
-		return nil, fmt.Errorf("failed to open zip archive %s: %v", archivePath, err)
+	// First try RAM cache and promote if found
+	if value, ok := c.ramCache.Get(key); ok {
+		if entry, ok := value.(*cacheEntry); ok {
+			return entry.content, nil
+		}
 	}
 
-	defer closeReader(r)
+	// If not in RAM cache, check disk cache
+	var content []byte
+	var timestamp []byte
+	err := c.db.Update(func(txn *badger.Txn) error {
+		// Create the entry key prefix
+		entryKeyPrefix := make([]byte, 1+len(diskKey))
+		entryKeyPrefix[0] = entryTag
+		copy(entryKeyPrefix[1:], diskKey)
+
+		// Create new byte slices for each key to avoid array modification issues
+		dataKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(dataKey, entryKeyPrefix)
+		dataKey[len(entryKeyPrefix)] = entrySubTagData
+
+		timestampKey := make([]byte, len(entryKeyPrefix)+1)
+		copy(timestampKey, entryKeyPrefix)
+		timestampKey[len(entryKeyPrefix)] = entrySubTagTime
+
+		// Get the data
+		item, err := txn.Get(dataKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("file %s not found in cache for website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-	for _, f := range r.File {
-		if f.Name == resourceName {
-			rc, err := f.Open()
-			if err != nil {
-				return nil, fmt.Errorf("failed to open file %s in zip: %v", f.Name, err)
-			}
+		content, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			defer closeReader(rc)
+		// Get the timestamp
+		item, err = txn.Get(timestampKey)
+		if err == badger.ErrKeyNotFound {
+			return fmt.Errorf("timestamp not found for file %s in website %s", resourceName, websiteAddress)
+		}
+		if err != nil {
+			return err
+		}
 
-			buf := new(bytes.Buffer)
+		timestamp, err = item.ValueCopy(nil)
+		if err != nil {
+			return err
+		}
 
-			_, err = io.Copy(buf, rc)
-			if err != nil {
-				return nil, fmt.Errorf("failed to copy content of file %s: %v", f.Name, err)
-			}
+		// Remove from disk
+		return c.deleteEntryFromDisk(txn, diskKey)
+	})
 
-			return buf.Bytes(), nil
-		}
+	if err != nil {
+		return nil, err
 	}
 
-	return nil, fmt.Errorf("file %s not found in archive %s", resourceName, archivePath)
-}
+	// Create a cache entry
+	entry := &cacheEntry{
+		content:        content,
+		modified:       time.Unix(0, int64(binary.BigEndian.Uint64(timestamp))),
+		websiteAddress: websiteAddress,
+		resourceName:   resourceName,
+	}
 
-// Save a resource in the cache for a given website
-func (c *Cache) Save(websiteAddress string, resourceName string, content []byte) error {
-	archivePath := c.getArchivePath(websiteAddress)
-	fileLock := c.newFileLock(archivePath)
-	tempArchivePath := archivePath + ".tmp"
+	// Add to RAM cache - eviction will be handled automatically by the callback
+	c.ramCache.Add(key, entry)
 
-	err := writeLockFile(fileLock)
+	return content, nil
+}
+
+// deleteEntryFromDisk removes an entry from the disk cache
+func (c *Cache) deleteEntryFromDisk(txn *badger.Txn, key []byte) error {
+	// Create the entry key prefix
+	entryKeyPrefix := make([]byte, 1+len(key))
+	entryKeyPrefix[0] = entryTag
+	copy(entryKeyPrefix[1:], key)
+
+	// Get the ID counter for this entry
+	idCounterKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(idCounterKey, entryKeyPrefix)
+	idCounterKey[len(entryKeyPrefix)] = entrySubTagID
+
+	item, err := txn.Get(idCounterKey)
+	if err == badger.ErrKeyNotFound {
+		return nil // Entry not found in disk cache
+	}
 	if err != nil {
-		//nolint:wrapcheck
 		return err
 	}
 
-	defer lockUnlock(fileLock)
-
-	var buf bytes.Buffer
-	w := zip.NewWriter(&buf)
-
-	err = c.copyExistingEntries(archivePath, w, resourceName)
+	idCounterValue, err := item.ValueCopy(nil)
 	if err != nil {
 		return err
 	}
 
-	header := &zip.FileHeader{
-		Name:     resourceName,
-		Method:   zip.Deflate,
-		Modified: time.Now(),
-	}
+	// Create data and timestamp keys
+	dataKey := make([]byte, len(entryKeyPrefix)+1)
+	copy(dataKey, entryKeyPrefix)
+	dataKey[len(entryKeyPrefix)] = entrySubTagData
path: server/pkg/cache/cache.go
user: login: damip
body: more info here https://github.com/hypermodeinc/badger/blob/62c6dc2af0819cab2f1ad4e7a7e9adb07a640f89/txn.go#L401-L402
created_at: 2025-03-24T10:43:53Z
author_association: MEMBER
diff_hunk: @@ -82,6 +158,16 @@ func GetFilesPathList(
 	client *node.Client,
 	websiteAddress string,
 ) ([]string, error) {
+	// Try to get from cache first
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		// Convert map back to slice
+		result := make([]string, 0, len(files))
+		for file := range files {
+			result = append(result, file)
+		}
+		return result, nil
path: server/pkg/website/read.go
user: login: peterjah
body: strange because the `maps` library has been added to sdt in go 1.21.
Did you added `import "maps"`
created_at: 2025-03-24T10:59:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -82,6 +158,16 @@ func GetFilesPathList(
 	client *node.Client,
 	websiteAddress string,
 ) ([]string, error) {
+	// Try to get from cache first
+	if files, exists := globalFilePathListCache.get(websiteAddress); exists {
+		// Convert map back to slice
+		result := make([]string, 0, len(files))
+		for file := range files {
+			result = append(result, file)
+		}
+		return result, nil
path: server/pkg/website/read.go
user: login: damip
body: ```
pkg/website/read.go:165:10: cannot use maps.Keys(files) (value of type iter.Seq[string]) as []string value in return statement
```
created_at: 2025-03-24T11:11:10Z
author_association: MEMBER

PR comments:
user: login: peterjah
created_at: 2025-03-24T09:47:17Z
body: It seems Badger has already a built-in RAM cache thats can be used alongside with encryption and compression of the disk data.
Maybe it would have been a cleaner way to use it...
author_association: CONTRIBUTOR
user: login: damip
created_at: 2025-03-24T10:40:54Z
body: > Well done! I think the code could be a bit more maintainable by splitting code (especially code of ram cache and badger), factorizing some reused logic and wrapping it into well named functions

Thanks for the very helpful review!
I split off the badger stuff into a separate file.


> Badger has already a built-in RAM cache thats can be used alongside with encryption and compression of the disk data.
> Maybe it would have been a cleaner way to use it...

I can't find details on that caching and how it works exactly. Do you have a link?
author_association: MEMBER
user: login: damip
created_at: 2025-03-24T10:49:50Z
body: Also fixed the lint.
author_association: MEMBER
user: login: peterjah
created_at: 2025-03-24T11:02:22Z
body: > > Well done! I think the code could be a bit more maintainable by splitting code (especially code of ram cache and badger), factorizing some reused logic and wrapping it into well named functions
> 
> Thanks for the very helpful review! I split off the badger stuff into a separate file.
> 
> > Badger has already a built-in RAM cache thats can be used alongside with encryption and compression of the disk data.
> > Maybe it would have been a cleaner way to use it...
> 
> I can't find details on that caching and how it works exactly. Do you have a link?

I was not able to find much documentation about it :/ 

https://pkg.go.dev/github.com/dgraph-io/badger/v3#Options.WithBlockCacheSize

It is stated here, that cache is useless if compression or encryption is not activated... so its not clear what it does ^^
author_association: CONTRIBUTOR
user: login: damip
created_at: 2025-03-24T11:11:48Z
body: Ok :) then let's leave the caching as is for better control
author_association: MEMBER