antirez/redis — Issue #367: Redis Data Loss
Author: gogzor
Created: 2012-03-01 09:49:25+00:00
State: open

We are using Node.JS in combination with Redis and we are having data loss issue. We have statistics collection system, so we must collect each request dealt to our server. With high concurrent requests (avg 1000/sec) we have data loss problems in Redis.

We have logged every request in Node.JS & Redis, the problem is that Node.JS shows exact the same amount of requests that were sent, Redis Monitor debugs the same amount, but Redis DBSIZE shows to us incorrect value. It is always ~40%-50% less than expected.

Our sever configuration is :

Intel® Core™ i7-920
24 GB DDR3 RAM
From Hetzner

We are running Debian Squeeze, here is our sysctl:

net.ipv4.conf.all.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1
net.core.somaxconn = 1280
fs.file-max = 50000
vm.overcommit_memory=1

We have "load average" of 0.04, and plenty of free memory (~20gb).

Any help regarding this issue will be highly appreciated.