Repository: LMCache/LMCache
Pull request #430: Introduce mooncakestore remote backend
Author: login: maobaolong
Created: 2025-03-28T12:30:18Z | Merged: 2025-04-16T14:58:47Z

# Description

FIX #429 

As mooncake store is a new coming and well known kv store, and LMCache is a mature kv connector implementation of vllm which can maintain variety of remote backends(e.g. redis, infinistore, lmserver), so it is a good choice to integrated mooncake store to lmcache.

# How to Test
```
# Start etcd
docker run  -p 2379:2379 -p 2380:2380 --rm -e ALLOW_NONE_AUTHENTICATION=yes  --name etcd2  bitnami/etcd
...
# Start mooncake_master
/mooncake/mooncake-store/src/mooncake_master -v=1 -port=50051

# Start vllm with lmcache connector and specific mooncake store as remote backend of lmcache
VLLM_USE_V1=0 \
MOONCAKE_CONFIG_PATH=./mooncake.json \
LMCACHE_USE_EXPERIMENTAL=True LMCACHE_TRACK_USAGE=false \
LMCACHE_CHUNK_SIZE=16 LMCACHE_LOCAL_CPU=False LMCACHE_MAX_LOCAL_CPU_SIZE=5.0 \
LMCACHE_REMOTE_URL=mooncakestore://localhost:50051/ \
LMCACHE_REMOTE_SERDE="cachegen" \
vllm serve /disc/f/models/opt-125m/ \
           --served-model-name "facebook/opt-125m" \
           --enforce-eager  \
           --port 8000 \
           --gpu-memory-utilization 0.8 \
           --kv-transfer-config '{"kv_connector":"LMCacheConnector","kv_role":"kv_both","kv_parallel_size":2}' \
           --trust-remote-code
           
```

- An example config of mooncake.json

```json
{
    "local_hostname": "localhost",
    "metadata_server": "etcd://localhost:2379",
    "protocol": "tcp",
    "device_name": "",
    "master_server_address": "localhost:50051"
}
```

- Tested by curl twice, one for cold cache miss, two for hot cache hit.

```
root@docker-desktop:/vllm-workspace# curl http://localhost:8000/v1/completions     -H "Content-Type: application/json"     -d '{
        "model": "facebook/opt-125m",
        "prompt": "San Francisco is a 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15",
        "max_tokens": 10,
        "temperature": 0
    }'
{"id":"cmpl-e9a422dd7e1b41afa58c2c666edd80a5","object":"text_completion","created":1744729807,"model":"facebook/opt-125m","choices":[{"index":0,"text":" 16 17 18 19 20 21 22 23 24 25","logprobs":null,"finish_reason":"length","stop_reason":null,"prompt_logprobs":null}],"usage":{"prompt_tokens":20,"total_tokens":30,"completion_tokens":10,"prompt_tokens_details":null}}root@docker-desktop:/vllm-workspace#
root@docker-desktop:/vllm-workspace#
root@docker-desktop:/vllm-workspace#
root@docker-desktop:/vllm-workspace#
root@docker-desktop:/vllm-workspace# curl http://localhost:8000/v1/completions     -H "Content-Type: application/json"     -d '{
        "model": "facebook/opt-125m",
        "prompt": "San Francisco is a 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15",
        "max_tokens": 10,
        "temperature": 0
    }'
{"id":"cmpl-132df463ebca49949906792ad629a9b3","object":"text_completion","created":1744729824,"model":"facebook/opt-125m","choices":[{"index":0,"text":" 16 17 18 19 20 21 22 23 24 25","logprobs":null,"finish_reason":"length","stop_reason":null,"prompt_logprobs":null}],"usage":{"prompt_tokens":20,"total_tokens":30,"completion_tokens":10,"prompt_tokens_details":null}}
```


- mooncake server log
```
root@docker-desktop:/disc/f/v1connector# /mooncake//mooncake-store/src/mooncake_master -v=1
I0415 22:08:38.866011  6048 master.cpp:250] Starting Mooncake Master Service
I0415 22:08:38.866413  6048 master.cpp:251] Port: 50051
I0415 22:08:38.866420  6048 master.cpp:252] Max threads: 4
I0415 22:08:38.867682  6048 master_service.cpp:97] action=gc_disabled
I0415 22:08:38.867739  6048 master.cpp:266] Garbage collection: disabled
I0415 22:08:38.869952  6048 master.cpp:292] Master service listening on 0.0.0.0:50051
I0415 22:09:15.189324  6052 master_service.cpp:125] segment_name=localhost:12461, buffer=140466804752384, size=3355443200, action=mount_segment
I0415 22:09:15.189370  6052 allocator.cpp:36] initializing_buffer_allocator segment_name=localhost:12461 base_address=0x7fc0fa000000 size=3355443200
I0415 22:09:15.189435  6052 allocator.cpp:62] buffer_allocator_initialized pool_id=0
I0415 22:09:15.189460  6052 master_service.cpp:63] segment_name=localhost:12461, base=140466804752384, size=3355443200, allocator_ptr=0x7f8a5c001580, action=register_buffer





I0415 23:10:07.894994  6189 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=get_replica_list_start
I0415 23:10:07.895036  6189 master_service.cpp:141] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, info=object_not_found
I0415 23:10:07.969734  6052 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=get_replica_list_start
I0415 23:10:07.969789  6052 master_service.cpp:141] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, info=object_not_found
I0415 23:10:07.971552  6189 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=get_replica_list_start
I0415 23:10:07.971593  6189 master_service.cpp:141] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, info=object_not_found
I0415 23:10:08.156286  6052 master_service.cpp:199] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, value_length=28, slice_count=1, config=ReplicateConfig: { replica_num: 1 }, action=put_start_begin
I0415 23:10:08.156337  6052 allocator.cpp:87] allocation_succeeded size=28 segment=localhost:12461 address=0x7fc0fa000000
I0415 23:10:08.156343  6052 allocator.cpp:16] buf_handle_created segment_id=0 size=28 buffer_address=0x7fc0fa000000
I0415 23:10:08.156347  6052 master_service.cpp:249] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, replica_id=0, slice_index=0, handle=BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 28, status: INIT, buffer: 0x7fc0fa000000 }, action=slice_allocated
I0415 23:10:08.156718  6052 master_service.cpp:259] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, replica_count=1, slice_count=1, action=put_start_complete
I0415 23:10:08.161164  6189 master_service.cpp:266] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=put_end_start
I0415 23:10:08.161206  6189 master_service.cpp:288] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=put_end_complete
I0415 23:10:08.164846  6052 master_service.cpp:199] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, value_length=1383097, slice_count=1, config=ReplicateConfig: { replica_num: 1 }, action=put_start_begin
I0415 23:10:08.164891  6052 allocator.cpp:87] allocation_succeeded size=1383097 segment=localhost:12461 address=0x7fc0fb000000
I0415 23:10:08.164916  6052 allocator.cpp:16] buf_handle_created segment_id=0 size=1383097 buffer_address=0x7fc0fb000000
I0415 23:10:08.164940  6052 master_service.cpp:249] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, replica_id=0, slice_index=0, handle=BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 1383097, status: INIT, buffer: 0x7fc0fb000000 }, action=slice_allocated
I0415 23:10:08.164968  6052 master_service.cpp:259] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, replica_count=1, slice_count=1, action=put_start_complete
I0415 23:10:08.167686  6189 master_service.cpp:266] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, action=put_end_start
I0415 23:10:08.167775  6189 master_service.cpp:288] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, action=put_end_complete
I0415 23:10:08.168709  6052 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, action=get_replica_list_start
I0415 23:10:08.168761  6052 master_service.cpp:141] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, info=object_not_found
I0415 23:10:08.183573  6189 master_service.cpp:199] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, value_length=28, slice_count=1, config=ReplicateConfig: { replica_num: 1 }, action=put_start_begin
I0415 23:10:08.183634  6189 allocator.cpp:87] allocation_succeeded size=28 segment=localhost:12461 address=0x7fc0fa000048
I0415 23:10:08.183642  6189 allocator.cpp:16] buf_handle_created segment_id=0 size=28 buffer_address=0x7fc0fa000048
I0415 23:10:08.183668  6189 master_service.cpp:249] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, replica_id=0, slice_index=0, handle=BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 28, status: INIT, buffer: 0x7fc0fa000048 }, action=slice_allocated
I0415 23:10:08.183681  6189 master_service.cpp:259] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, replica_count=1, slice_count=1, action=put_start_complete
I0415 23:10:08.184497  6189 master_service.cpp:266] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, action=put_end_start
I0415 23:10:08.184559  6189 master_service.cpp:288] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, action=put_end_complete
I0415 23:10:08.185388  6052 master_service.cpp:199] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, value_length=1311759, slice_count=1, config=ReplicateConfig: { replica_num: 1 }, action=put_start_begin
I0415 23:10:08.185443  6052 allocator.cpp:87] allocation_succeeded size=1311759 segment=localhost:12461 address=0x7fc0fb16fbb8
I0415 23:10:08.185451  6052 allocator.cpp:16] buf_handle_created segment_id=0 size=1311759 buffer_address=0x7fc0fb16fbb8
I0415 23:10:08.185456  6052 master_service.cpp:249] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, replica_id=0, slice_index=0, handle=BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 1311759, status: INIT, buffer: 0x7fc0fb16fbb8 }, action=slice_allocated
I0415 23:10:08.185508  6052 master_service.cpp:259] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, replica_count=1, slice_count=1, action=put_start_complete
I0415 23:10:08.189270  6189 master_service.cpp:266] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, action=put_end_start
I0415 23:10:08.189316  6189 master_service.cpp:288] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, action=put_end_complete













I0415 23:10:24.482215  6189 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=get_replica_list_start
I0415 23:10:24.482275  6189 master_service.cpp:156] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, replica_list=[ReplicaInfo: { replica_id: 0, status: COMPLETE, handles: [BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 28, status: COMPLETE, buffer: 0x7fc0fa000000 }] }]
I0415 23:10:24.486477  6052 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, action=get_replica_list_start
I0415 23:10:24.486522  6052 master_service.cpp:156] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04kv_bytes, replica_list=[ReplicaInfo: { replica_id: 0, status: COMPLETE, handles: [BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 1383097, status: COMPLETE, buffer: 0x7fc0fb000000 }] }]
I0415 23:10:24.516680  6189 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, action=get_replica_list_start
I0415 23:10:24.516710  6189 master_service.cpp:156] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, replica_list=[ReplicaInfo: { replica_id: 0, status: COMPLETE, handles: [BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 28, status: COMPLETE, buffer: 0x7fc0fa000048 }] }]
I0415 23:10:24.517853  6052 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, action=get_replica_list_start
I0415 23:10:24.517905  6052 master_service.cpp:156] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74kv_bytes, replica_list=[ReplicaInfo: { replica_id: 0, status: COMPLETE, handles: [BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 1311759, status: COMPLETE, buffer: 0x7fc0fb16fbb8 }] }]
I0415 23:10:24.612699  6189 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, action=get_replica_list_start
I0415 23:10:24.612749  6189 master_service.cpp:156] key=vllm@/disc/f/models/opt-125m/@1@0@eda2c6ce5d62f6ec62ef2af890747376dad26c0e134be6f50ee350f3e5288c04metadata, replica_list=[ReplicaInfo: { replica_id: 0, status: COMPLETE, handles: [BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 28, status: COMPLETE, buffer: 0x7fc0fa000000 }] }]
I0415 23:10:24.613636  6052 master_service.cpp:137] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, action=get_replica_list_start
I0415 23:10:24.613695  6052 master_service.cpp:156] key=vllm@/disc/f/models/opt-125m/@1@0@0090a08f5da272f432b977a4b5e9c8b8072d07c5f283a13d29ba83097e344e74metadata, replica_list=[ReplicaInfo: { replica_id: 0, status: COMPLETE, handles: [BufHandle: { segment_id: 0, segment_name: localhost:12461, size: 28, status: COMPLETE, buffer: 0x7fc0fa000048 }] }]

```




Linked issue #429: Introduce mooncake store as a remote backend of LMCache
As title, make mooncake store run as a remote backend of LMCache
Linked issue comments: user: login: YaoJiayi
created_at: 2025-03-28T15:20:06Z
body: @maobaolong Hi, mooncake is definitely on our roadmap!
author_association: COLLABORATOR
user: login: maobaolong
created_at: 2025-03-29T04:19:07Z
body: @YaoJiayi Thanks for your reply, I've submit a PR as a startup.
author_association: COLLABORATOR
user: login: WangJunliang1
created_at: 2025-05-09T01:29:12Z
body: > [@YaoJiayi](https://github.com/YaoJiayi) Thanks for your reply, I've submit a PR as a startup.

Thanks for your work. I noticed that kv_role is set to "kv_both"; is there any way to achieve a LMCache Connector + Mooncake-Store based approach to PD disaggregation?
author_association: NONE
user: login: chngalx
created_at: 2025-05-13T08:35:30Z
body: Will LMcache + MoonCake-store support xPyD?
author_association: NONE

Diff:
diff --git a/examples/experimental/storage_backends/mooncakestore/README.md b/examples/experimental/storage_backends/mooncakestore/README.md
new file mode 100644
index 0000000000..bcded70281
--- /dev/null
+++ b/examples/experimental/storage_backends/mooncakestore/README.md
@@ -0,0 +1,47 @@
+lmcache could use [mooncakestore](https://github.com/kvcache-ai/Mooncake) as a backend storage.
+
+Mooncakestore is a memory storage which support RDMA and TCP. lmcache's mooncakestore connector can use TCP/RDMA transport for now.
+
+This is a simple instruction how to use lmcache with mooncakestore
+
+# install mooncakestore
+
+1. build mooncake and start mooncake store and dependent services.
+
+Please follow [build guild](https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md)
+
+2. install mooncakestore mooncake_vllm_adaptor 
+
+For now, you have to build the mooncake from source follow [build guild](https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md).
+But mooncake will supply pip install for nearly future.
+
+# start mooncake store and lmcache
+
+1. start mooncake store
+
+Reference [mooncake guide](https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/vllm-integration-v1.md#run-example)
+
+For example.
+
+```
+mooncake_master -v=1 -port=50051
+```
+
+2. start vllm with lmcache connector
+```
+VLLM_USE_V1=0 \
+MOONCAKE_CONFIG_PATH=./mooncake.json \
+LMCACHE_USE_EXPERIMENTAL=True LMCACHE_TRACK_USAGE=false \
+LMCACHE_CHUNK_SIZE=16 LMCACHE_LOCAL_CPU=False LMCACHE_MAX_LOCAL_CPU_SIZE=5.0 \
+LMCACHE_REMOTE_URL=mooncakestore://localhost:50051/ \
+LMCACHE_REMOTE_SERDE="cachegen" \
+vllm serve /disc/f/models/opt-125m/ \
+           --served-model-name "facebook/opt-125m" \
+           --enforce-eager  \
+           --port 8000 \
+           --gpu-memory-utilization 0.8 \
+           --kv-transfer-config '{"kv_connector":"LMCacheConnector","kv_role":"kv_both","kv_parallel_size":2}' \
+           --trust-remote-code
+```
+
+The `./mooncake.json` can reference [mooncake store config](https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/vllm-integration-v1.md#configuration)
diff --git a/examples/experimental/storage_backends/mooncakestore/backend_type.yaml b/examples/experimental/storage_backends/mooncakestore/backend_type.yaml
new file mode 100644
index 0000000000..1d2f40ef77
--- /dev/null
+++ b/examples/experimental/storage_backends/mooncakestore/backend_type.yaml
@@ -0,0 +1,2 @@
+chunk_size: 256
+remote_url: "mooncakestore://127.0.0.1:50051"
diff --git a/lmcache/experimental/storage_backend/connector/__init__.py b/lmcache/experimental/storage_backend/connector/__init__.py
index 07fb3990aa..bbb2b29ec6 100644
--- a/lmcache/experimental/storage_backend/connector/__init__.py
+++ b/lmcache/experimental/storage_backend/connector/__init__.py
@@ -27,6 +27,7 @@
 from lmcache.logging import init_logger
 
 from .infinistore_connector import InfinistoreConnector
+from .mooncakestore_connector import MooncakestoreConnector
 
 logger = init_logger(__name__)
 
@@ -159,6 +160,11 @@ def CreateConnector(
             device_name = parsed_url.query_params[0].get("device", "mlx5_0")
             connector = InfinistoreConnector(host, port, device_name, loop,
                                              memory_allocator)
+        case "mooncakestore":
+            host, port = parsed_url.hosts[0], parsed_url.ports[0]
+            device_name = parsed_url.query_params[0].get("device", "")
+            connector = MooncakestoreConnector(host, port, device_name, loop,
+                                               memory_allocator)
         case _:
             raise ValueError(
                 f"Unknown connector type {parsed_url.connector_type} "
diff --git a/lmcache/experimental/storage_backend/connector/mooncakestore_connector.py b/lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
new file mode 100644
index 0000000000..bb828c3165
--- /dev/null
+++ b/lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
@@ -0,0 +1,172 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None:
+            return None
+
+        assert not inspect.isawaitable(metadata_bytes)
+
+        metadata = RedisMetadata.deserialize(memoryview(metadata_bytes))
+
+        memory_obj = self.memory_allocator.allocate(
+            metadata.shape,
+            metadata.dtype,
+            metadata.fmt,
+        )
+        if memory_obj is None:
+            logger.warning("Failed to allocate memory during remote receive")
+            return None
+
+        kv_bytes = self.store.get(key_str + "kv_bytes")
+        assert not inspect.isawaitable(kv_bytes)
+        assert kv_bytes is not None
+
+        view = memoryview(memory_obj.byte_array)
+        view[:metadata.length] = kv_bytes
+
+        return memory_obj
+
+    async def put(self, key: CacheEngineKey, memory_obj: MemoryObj):
+        # Please use a function like `memory_obj.to_meta()`.
+        kv_bytes = memory_obj.byte_array
+        kv_shape = memory_obj.get_shape()
+        kv_dtype = memory_obj.get_dtype()
+        memory_format = memory_obj.get_memory_format()
+
+        metadata_bytes = RedisMetadata(len(kv_bytes), kv_shape, kv_dtype,
+                                       memory_format).serialize()
+
+        key_str = key.to_string()
+        self.store.put(key_str + "metadata", metadata_bytes)
+        key_byte_str = key_str + "kv_bytes"
+        try:
+            self.store.put(key_byte_str, kv_bytes)
+        except Exception as e:
+            logger.error(f"Failed to put key {key_byte_str},"
+                         f"meta type: {type(metadata_bytes)},"
+                         f"data: {type(kv_bytes)}: {e}")
+
+        self.memory_allocator.ref_count_down(memory_obj)
+
+    @no_type_check
+    async def list(self) -> List[str]:
+        pass
+
+    async def close(self):
+        self.store.close()
+        logger.info("Closed the mooncake store connection")


Reviews:
user: login: YaoJiayi
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:17:50Z
user: login: YaoJiayi
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:18:23Z
user: login: YaoJiayi
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:20:12Z
user: login: YaoJiayi
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:20:53Z
user: login: YaoJiayi
body: Thanks @maobaolong for the PR! I left some comments:)
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:22:15Z
user: login: maobaolong
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:25:09Z
user: login: maobaolong
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:25:21Z
user: login: maobaolong
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:27:12Z
user: login: maobaolong
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-15T16:29:37Z
user: login: YaoJiayi
body: Took another look. Just have one pending comment:)
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-16T06:04:11Z
user: login: maobaolong
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-16T10:54:40Z
user: login: YaoJiayi
body: lgtm!
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-16T14:58:42Z

Review comments:
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
+            return None
+
+        assert not inspect.isawaitable(metadata_bytes)
+
+        metadata = RedisMetadata.deserialize(memoryview(metadata_bytes))
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: YaoJiayi
body: The naming of `RedisMetadata` is a bit weird here
created_at: 2025-04-15T16:17:50Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: YaoJiayi
body: When is this buffer used?
created_at: 2025-04-15T16:18:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: YaoJiayi
body: When is this used?
created_at: 2025-04-15T16:20:12Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: YaoJiayi
body: When does condition `len(metadata_bytes) != METADATA_BYTES_LEN` hold?
created_at: 2025-04-15T16:20:53Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: maobaolong
body: Sorry, it is an intermediate code, remove it.
created_at: 2025-04-15T16:25:09Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: maobaolong
body: Remove it
created_at: 2025-04-15T16:25:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
+            return None
+
+        assert not inspect.isawaitable(metadata_bytes)
+
+        metadata = RedisMetadata.deserialize(memoryview(metadata_bytes))
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: maobaolong
body: Yeah, I just need `deserialize` from `RedisMetadata`, could you help to give me some suggestion? In fact, I just follow what infinistore_connector did.
created_at: 2025-04-15T16:27:12Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: maobaolong
body: It could hold during my development, since I put the wrong type metadata into mooncake store. 
created_at: 2025-04-15T16:29:36Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: YaoJiayi
body: Sorry I do not get this. Why are we putting the wrong metadata?
created_at: 2025-04-16T06:03:25Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
+            return None
+
+        assert not inspect.isawaitable(metadata_bytes)
+
+        metadata = RedisMetadata.deserialize(memoryview(metadata_bytes))
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: YaoJiayi
body: We can keep it this way for now:)
created_at: 2025-04-16T06:03:44Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+# Copyright 2024-2025 LMCache Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import asyncio
+import ctypes
+import inspect
+import json
+import os
+from dataclasses import dataclass
+from typing import List, Optional, Union, no_type_check
+
+from lmcache.experimental.memory_management import (MemoryAllocatorInterface,
+                                                    MemoryObj)
+# reuse
+from lmcache.experimental.protocol import RedisMetadata
+from lmcache.experimental.storage_backend.connector.base_connector import \
+    RemoteConnector
+from lmcache.logging import init_logger
+from lmcache.utils import CacheEngineKey
+
+logger = init_logger(__name__)
+
+METADATA_BYTES_LEN = 28
+
+
+def _get_ptr(mv: Union[bytearray, memoryview]) -> int:
+    return ctypes.addressof(ctypes.c_char.from_buffer(mv))
+
+
+@dataclass
+class MooncakeStoreConfig:
+    local_hostname: str
+    metadata_server: str
+    global_segment_size: int
+    local_buffer_size: int
+    protocol: str
+    device_name: str
+    master_server_address: str
+
+    @staticmethod
+    def from_file(file_path: str) -> 'MooncakeStoreConfig':
+        """Load the config from a JSON file."""
+        with open(file_path) as fin:
+            config = json.load(fin)
+        return MooncakeStoreConfig(
+            local_hostname=config.get("local_hostname"),
+            metadata_server=config.get("metadata_server"),
+            global_segment_size=config.get("global_segment_size", 3355443200),
+            local_buffer_size=config.get("local_buffer_size", 1073741824),
+            protocol=config.get("protocol", "tcp"),
+            device_name=config.get("device_name", ""),
+            master_server_address=config.get("master_server_address"),
+        )
+
+    @staticmethod
+    def load_from_env() -> 'MooncakeStoreConfig':
+        """Load config from a file specified in the environment variable."""
+        config_file_path = os.getenv('MOONCAKE_CONFIG_PATH')
+        if config_file_path is None:
+            raise ValueError(
+                "The environment variable 'MOONCAKE_CONFIG_PATH' is not set.")
+        return MooncakeStoreConfig.from_file(config_file_path)
+
+
+class MooncakestoreConnector(RemoteConnector):
+
+    def __init__(self, host: str, port: int, dev_name,
+                 loop: asyncio.AbstractEventLoop,
+                 memory_allocator: MemoryAllocatorInterface):
+        try:
+            from mooncake_vllm_adaptor import MooncakeDistributedStore
+        except ImportError as e:
+            raise ImportError(
+                "Please install mooncake by following the instructions at "
+                "https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md "  # noqa: E501
+                "to run vLLM with MooncakeConnector.") from e
+
+        try:
+            self.store = MooncakeDistributedStore()
+            self.config = MooncakeStoreConfig.load_from_env()
+            if host != "" and port != 0:
+                self.config.master_server_address = host + ":" + str(port)
+            if dev_name != "":
+                self.config.device_name = dev_name
+            logger.info("Mooncake Configuration loaded. config: %s",
+                        self.config)
+
+            self.store.setup(self.config.local_hostname,
+                             self.config.metadata_server,
+                             self.config.global_segment_size,
+                             self.config.local_buffer_size,
+                             self.config.protocol, self.config.device_name,
+                             self.config.master_server_address)
+
+        except ValueError as e:
+            logger.error("Configuration loading failed: %s", e)
+            raise
+        except Exception as exc:
+            logger.error(
+                "An error occurred while loading the configuration: %s", exc)
+            raise
+
+        self.memory_allocator = memory_allocator
+        self.loop = loop
+
+        # allocate 4KB buffer for RDMA read
+        self.buffer_size = 4 << 10
+        self.buffer = bytearray(self.buffer_size)
+
+    async def exists(self, key: CacheEngineKey) -> bool:
+        return self.store.isExist(key.to_string() + "metadata")
+
+    async def get(self, key: CacheEngineKey) -> Optional[MemoryObj]:
+        key_str = key.to_string()
+
+        metadata_bytes = self.store.get(key_str + "metadata")
+        if metadata_bytes is None or len(metadata_bytes) != METADATA_BYTES_LEN:
path: lmcache/experimental/storage_backend/connector/mooncakestore_connector.py
user: login: maobaolong
body: @YaoJiayi Sorry to let you confusing, let me explain more.

Firstly, this could rarely happen in production.

I add this check since my own mistake while develop this feature, I just put a wrong metadata caused by a bug, so I add this check here. 

For make this clear, I'd like to remove this check.
created_at: 2025-04-16T10:54:40Z
author_association: COLLABORATOR

PR comments:
user: login: maobaolong
created_at: 2025-04-15T15:43:34Z
body: @ApostaC @YaoJiayi Would you like to take a look at this PR? Thanks.
author_association: COLLABORATOR
user: login: maobaolong
created_at: 2025-04-15T16:33:35Z
body: @YaoJiayi Thanks for your review! The comments have been addressed by the last commit, left a question inline. PTAL.
author_association: COLLABORATOR
user: login: maobaolong
created_at: 2025-04-16T10:57:18Z
body: @YaoJiayi Sorry for the confusing check, just remove it for now, and explained below your comment. PTAL
author_association: COLLABORATOR