groonga/groonga — pull request #120 by wm3 (2013-10-13 10:16:32+00:00)
Title: groonga-httpd: load command by POST method causes timeout

Issue
----------------

When I call load command to groonga-httpd using POST method, the server sometimes does not reply anything.

This timeout does not occurs when:

* load command is called by GET method,
* the data is small (<1kb),
* or the data is very large (>10kb).

Example
----------------

```sh
% sh structure.sh
+ curl 'http://localhost:10041/d/table_create?name=Records&key_type=Int64'
[[0,1381657055.32697,0.000480175018310547],true]+ curl 'http://localhost:10041/d/column_create?table=Records&name=digits&type=ShortText'
[[0,1381657055.33599,0.000612020492553711],true]%
% time ruby load.rb
1: 897316929176464ebc9ad085f31e7284
2: 0ade138937c4b9cb36a28e2edb6485fc
4: e5870c1091c20ed693976546d23b4841
8: d05f866c1ae07ed18de845cf2da68de5
16: 04f2f684118b5f3ad55c03ab12e96bf2
32: 4de4bee232a48bf49c7df42334e36da4
64: 5267c61d6bac1e00ed5e45235387b1ad
128: 39658d59911b076912e05d6256b80479
256: 92d0356e1fccd00381502718fb2b14b2
512: 653f503a044fbe0ba33bfadf3dd1b987
/var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:879:in `gets': execution expired (HTTPClient::ReceiveTimeoutError)
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:879:in `block in parse_header'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:875:in `parse_header'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:858:in `read_header'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:667:in `get_header'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1137:in `do_get_header'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1086:in `do_get_block'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:887:in `block in do_request'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:981:in `protect_keep_alive_disconnected'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:886:in `do_request'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:959:in `follow_redirect'
from /var/lib/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:631:in `post_content'
from load.rb:24:in `block in

'
from load.rb:20:in `each'
from load.rb:20:in `

'

real	1m0.117s
user	0m0.064s
sys	0m0.024s
```

```sh
#!/bin/sh

# structure.sh

set -ex

curl 'http://localhost:10041/d/table_create?name=Records&key_type=Int64'
curl 'http://localhost:10041/d/column_create?table=Records&name=digits&type=ShortText'
```

```
#!/usr/bin/ruby

# load.rb
#
# gem install httpclient --no-ri --no-rdoc

require 'json'
require 'httpclient'
require 'digest/md5'

def loading_data(*records)
[["_key", "digits"]] + records
end

def digit_1234_cycle(i)
block = "0123456789"
block * (i / 10) + block[0...(i % 10)]
end

print "%10s: %s\n" % [ "_key", "md5(digits)" ]
c = HTTPClient.new
(0..24).each do |i|

length = 2 ** i
data = digit_1234_cycle(length)
res = c.post_content("http://localhost:10041/d/load?table=Records",
:header => { 'Content-Type' => 'application/json' },
:body => loading_data([length, data]).to_json
)
print "%10d: %s\n" % [ length, Digest::MD5.hexdigest(data + "\n") ]
end
```

Observation
----------------

Groonga-httpd does not support request bodies that have multiple buffers.

It seems that request body has multiple buffers when entire request has larger than 1kb (client_header_buffer_size?) and the request body has smaller than about 10kb.

This PR supports requests that have multiple buffers by joining each byte of buffer.