antirez/redis — Issue #243: Crash with ASSERTION FAILED after longer uptime
Author: d1april
Created: 2011-12-11 02:30:23+00:00
State: open

I run a big ranking with redis but redis always crashes after some days of operation.
Thanks to version 2.4.4 I have something more detailed about it than just a signal: 11 crash reporting.

I hope someone can find our what this is. Do you need more info from me?

[10116] 10 Dec 17:44:28 # === REDIS BUG REPORT START: Cut & paste starting from here ===
[10116] 10 Dec 17:44:28 # === ASSERTION FAILED ===
[10116] 10 Dec 17:44:28 # ==> t_zset.c:2043 'rank' is not true
[10116] 10 Dec 17:44:28 # (forcing SIGSEGV to print the bug report.)
[10116] 10 Dec 17:44:28 # Redis 2.4.4 crashed by signal: 11
[10116] 10 Dec 17:44:28 # Failed assertion: rank (t_zset.c:2043)
[10116] 10 Dec 17:44:28 # --- STACK TRACE
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(_redisAssert+0x6f) [0x431d6f]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(_redisAssert+0x6f) [0x431d6f]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(zrankGenericCommand+0x259) [0x427fc9]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(call+0x34) [0x410784]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(processCommand+0x26c) [0x410acc]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(processInputBuffer+0x4f) [0x419b1f]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(readQueryFromClient+0x6c) [0x419bfc]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(aeProcessEvents+0x153) [0x40d433]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(aeMain+0x2e) [0x40d67e]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server(main+0xff) [0x4128ff]
[10116] 10 Dec 17:44:28 # /lib/libc.so.6(__libc_start_main+0xfd) [0x7fad195a7c4d]
[10116] 10 Dec 17:44:28 # /usr/local/bin/redis-server() [0x40c989]
[10116] 10 Dec 17:44:28 # --- INFO OUTPUT
[10116] 10 Dec 17:44:28 # redis_version:2.4.4M
redis_git_sha1:00000000M
redis_git_dirty:0M
arch_bits:64M
multiplexing_api:epollM
process_id:10116M
uptime_in_seconds:527463M
uptime_in_days:6M
lru_clock:232970M
used_cpu_sys:1801.33M
used_cpu_user:1398.54M
used_cpu_sys_children:1088.90M
used_cpu_user_children:19349.28M
connected_clients:4M
connected_slaves:0M
client_longest_output_list:0M
client_biggest_input_buf:0M
blocked_clients:0M
used_memory:1494171944M
used_memory_human:1.39GM
used_memory_rss:1501999104M
used_memory_peak:1494171816M
used_memory_peak_human:1.39GM
mem_fragmentation_ratio:1.01M
mem_allocator:jemalloc-2.2.5M
loading:0M
aof_enabled:0M
changes_since_last_save:8015M
bgsave_in_progress:0M
last_save_time:1323535184M
bgrewriteaof_in_progress:0M
total_connections_received:2120M
total_commands_processed:124112523M
expired_keys:0M
evicted_keys:0M
keyspace_hits:123622134M
keyspace_misses:488270M
pubsub_channels:0M
pubsub_patterns:0M
latest_fork_usec:43677M
vm_enabled:0M
role:masterM
db0:keys=81,expires=0M

[10116] 10 Dec 17:44:28 # --- CLIENT LIST OUTPUT
[10116] 10 Dec 17:44:28 # addr=127.0.0.1:39284 fd=8 idle=0 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=zadd
addr=127.0.0.1:59485 fd=5 idle=0 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=zrevrank
addr=127.0.0.1:35242 fd=7 idle=1 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=zcard
addr=127.0.0.1:51220 fd=6 idle=17 flags=N db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=zcard

[10116] 10 Dec 17:44:28 # === REDIS BUG REPORT END. Make sure to include from START to END. ===

For the next time it crashes I try to log what happens on client side, if it's even recognized by the client and what the last communication with redis was.