bitly/nsq — pull request #216 by jehiah (2013-06-08 17:27:20+00:00)
Title: nsq_to_file: corrupt empty gzip file

I ran into a case where nsq_to_file opened a new gzip file, but never flushed any messages to that file. As a result it didn't get a gzip frame and was corrupt.

One might expect in situations like this that a file shouldn't be opened at all, or that it should have a correct empty gzip header.

nsq_to_file output. Note the file was opened, no messages were flushed to it before a clean shutdown.
```
2013/06/05 20:00:04 opening /data/realtime/stream_archive/events.localhost.2013-06-05_20.log.gz
2013/06/05 20:00:04 [nsqd03:4150] received heartbeat from nsqd
2013/06/05 20:00:04 [nsqd04:4150] received heartbeat from nsqd
2013/06/05 20:00:34 [nsqd03:4150] received heartbeat from nsqd
2013/06/05 20:00:34 [nsqd04:4150] received heartbeat from nsqd
2013/06/05 20:00:38 Stopping reader
2013/06/05 20:00:38 [nsqd04:4150] received ACK from nsqd - now in CLOSE_WAIT
2013/06/05 20:00:38 [nsqd04:4150] beginning stopFinishLoop logic
2013/06/05 20:00:38 [nsqd04:4150] breaking out of finish loop
2013/06/05 20:00:38 [nsqd04:4150] finishLoop exiting
2013/06/05 20:00:38 there are 1 connections left alive
2013/06/05 20:00:38 [nsqd04:4150] stopped read loop
2013/06/05 20:00:38 [nsqd04:4150] readLoop exiting
2013/06/05 20:00:38 [nsqd04:4150] rdyLoop exiting
2013/06/05 20:00:38 [nsqd03:4150] received ACK from nsqd - now in CLOSE_WAIT
2013/06/05 20:00:38 [nsqd03:4150] beginning stopFinishLoop logic
2013/06/05 20:00:38 [nsqd03:4150] breaking out of finish loop
2013/06/05 20:00:38 [nsqd03:4150] finishLoop exiting
2013/06/05 20:00:38 there are 0 connections left alive
2013/06/05 20:00:38 closing incomingMessages
2013/06/05 20:00:38 [nsqd03:4150] stopped read loop
2013/06/05 20:00:38 [nsqd03:4150] readLoop exiting
2013/06/05 20:00:38 closing AsyncHandler (after self.incomingMessages closed)
2013/06/05 20:00:38 [nsqd03:4150] rdyLoop exiting
2013/06/05 20:00:39 starting AsyncHandler go-routine
```

```
$ zcat /data/realtime/stream_archive/events.localhost.2013-06-05_20.log.gz | wc -l

zcat: /data/realtime/stream_archive/events.localhost.2013-06-05_20.log.gz: unexpected end of file
0
```