yesodweb/wai — pull request #84 by tsuraan (2012-06-13 23:19:21+00:00)
Title: wai-extra form processing speedup

When using wai-extra's parseRequestBody to handle a file upload form, the findBound function takes a lot of CPU time. Here's a simple (non-form-handling) app that can process large amounts of data:

```
application req = do
ctx <- requestBody req $$ CondList.fold Tiger.update Tiger.init
let hash = fromByteString $ hex $ Tiger.finalize ctx
return $ ResponseBuilder status200 [("Content-Type", "text/plain")] hash
```
This responds to the user with the tiger hash of the stream they uploaded. I made a test file with ```dd if=/dev/zero of=zeroes bs=1M count=1 seek=2048``` and uploaded it with ```time curl --data-binary @zeroes http://localhost:3000```. The result was

```
46D9CEC9F09ACD2D800B41B1E72DC568C4CE826E9A965151
real 0m11.694s
user 0m1.058s
sys 0m4.562s
```
So, wai is fast when you just use the raw conduit. A modification of this web server to handle form uploads (and return "Hello world" instead of the hashes because I'm lazy) looks like this:

```
backend :: BackEnd ByteString
backend = BackEnd initialize append close finalize
where
initialize = return Tiger.init
append x bs = return $! Tiger.update x bs
close = return . hex . Tiger.finalize
finalize y = return ()

application req = do
(ps, fs) <- parseRequestBody backend req
liftIO $ print ps
liftIO $ print fs
return $ responseLBS status200 [("Content-Type", "text/plain")] "Hello world\n"
```
Using the current git head and the curl command ```time curl --form b=c --form c=d --form a=@zeroes http://localhost:3000```, I get the result:

```
Hello world

real 3m22.166s
user 0m1.792s
sys 0m8.057s
```
So, for basically the same effect, we went from 11s to 202s. Profiling shows that 94% of the time is spent in findBound, so I grabbed bos's Data.Text.Search module and hacked it a bit. The provided patch gives this result, for the same curl command:

```
time curl --form b=c --form c=d --form a=@zeroes http://localhost:3000
Hello world

real 0m13.727s
user 0m0.820s
sys 0m4.175s
```

Without the "-O2" change, the time was 14.8s, so a bit slower. I thought the -O2 worthwhile, but maybe it has consequences that I'm not aware of. Either way, just changing the findBound function was pretty good. My file hashes are all checking no matter which code I use, so I don't think I broke anything too badly. I think that things could be a little faster if I did something more clever with the ```Nothing``` side of Search.index, but I'm pretty happy with the results right now.