bitly/nsq — pull request #245 by mreiferson (2013-08-11 21:13:44+00:00)
Title: nsqd: panic on /stats when clients present

The stack trace below is what I get on the console when my instance of nsqd stops responding. The messages are plain text, not JSON being sent across the topics, have one channel attached to each affected topic.

I've been able to reproduce this on a sandbox server with one channel named "test" and one channel named "chan". Sending a couple of messages across the wire, then letting it sit for a while (~5 minutes), coming back and sending another one, causes the stats endpoint on nsqd to return empty results.

Total setup looks like this:

* nsqlookupd
* nsqd with tls and connected to lookupd
* nsqadmin connected to looked
* nsq_pubsub example with one subscriber on topic `test`, channel `chan`.

Ubuntu 13.04/amd64 3.8.0-23-generic SMP

```
2013/08/11 00:27:22 http: panic serving 186.4.15.88:36956: runtime error: index out of range
/usr/lib/go/src/pkg/net/http/server.go:576 (0x4ce692)
_func_003: buf.Write(debug.Stack())
/build/buildd/golang-1.0.2/src/pkg/runtime/proc.c:1443 (0x4338e5)
/build/buildd/golang-1.0.2/src/pkg/runtime/runtime.c:128 (0x4344c5)
/build/buildd/golang-1.0.2/src/pkg/runtime/runtime.c:85 (0x43436c)
/home/jtregunna/nsq/nsqd/stats.go:117 (0x41c22e)
(*NSQd).getStats: clients[client_index] = client.Stats()
/home/jtregunna/nsq/nsqd/http.go:407 (0x40cecc)
(*httpServer).statsHandler: stats := s.context.nsqd.getStats()
/home/jtregunna/nsq/nsqd/http.go:34 (0x40a53d)
(*httpServer).ServeHTTP: s.statsHandler(w, req)
/usr/lib/go/src/pkg/net/http/server.go:656 (0x4c24a4)
(*conn).serve: handler.ServeHTTP(w, w.req)
/build/buildd/golang-1.0.2/src/pkg/runtime/proc.c:271 (0x4319eb)
2013/08/11 00:27:25 LOOKUPD(picard.srcd.mp:4160): sending heartbeat
2013/08/11 00:27:25 http: panic serving 186.4.15.88:34498: runtime error: index out of range
/usr/lib/go/src/pkg/net/http/server.go:576 (0x4ce692)
_func_003: buf.Write(debug.Stack())
/build/buildd/golang-1.0.2/src/pkg/runtime/proc.c:1443 (0x4338e5)
/build/buildd/golang-1.0.2/src/pkg/runtime/runtime.c:128 (0x4344c5)
/build/buildd/golang-1.0.2/src/pkg/runtime/runtime.c:85 (0x43436c)
/home/jtregunna/nsq/nsqd/stats.go:117 (0x41c22e)
(*NSQd).getStats: clients[client_index] = client.Stats()
/home/jtregunna/nsq/nsqd/http.go:407 (0x40cecc)
(*httpServer).statsHandler: stats := s.context.nsqd.getStats()
/home/jtregunna/nsq/nsqd/http.go:34 (0x40a53d)
(*httpServer).ServeHTTP: s.statsHandler(w, req)
/usr/lib/go/src/pkg/net/http/server.go:656 (0x4c24a4)
(*conn).serve: handler.ServeHTTP(w, w.req)
/build/buildd/golang-1.0.2/src/pkg/runtime/proc.c:271 (0x4319eb)
```