mailrank/aeson — Issue #37: Space leak in `Data.Aeson.Parser`
Author: hvr
Created: 2011-11-03 11:03:11+00:00
State: open

This issue is mostly for documenting a known issue and a possible fix/workaround

The current parser implementation in `Data/Aeson/Parser` creates many thunks for records and arrays during parsing. Those thunks cause measurable overhead in heap usage and performance, which is already visible for some of the JSON documents contained in the Aeson benchmark suite. For even larger JSON documents in the MiB range, the overhead becomes even more substantial. Compared to Python's parser, Aeson then is a few times slower.

The benchmark suite itself doesn't evaluate those thunks, unless the following diff is applied:

```diff
diff --git a/benchmarks/AesonParse.hs b/benchmarks/AesonParse.hs
index a05ed5f..9e74e56 100644
--- a/benchmarks/AesonParse.hs
+++ b/benchmarks/AesonParse.hs
@@ -24,7 +24,9 @@ main = do
let refill = B.hGet h 16384
result <- parseWith refill json =<< refill
case result of
- Done _ r -> loop (good+1) bad
+ Done _ r -> do
+ evaluate $ rnf r
+ loop (good+1) bad
_ -> loop good (bad+1)
(good, _) <- loop 0 0
delta <- flip diffUTCTime start `fmap` getCurrentTime
```

The following modification removes the space-leak in `Data.Aeson.Parser`:

```diff
diff --git a/Data/Aeson/Parser.hs b/Data/Aeson/Parser.hs
index ba7b135..8bbe07e 100644
--- a/Data/Aeson/Parser.hs
+++ b/Data/Aeson/Parser.hs
@@ -55,19 +55,23 @@ object_ = {-# SCC "object_" #-} do
b <- char ':' *> skipSpace *> value
return (a,b)
vals <- ((pair <* skipSpace) `sepBy` (char ',' *> skipSpace)) <* char '}'
- return . Object $ Map.fromList vals
+ return $! Object $! Map.fromList vals

array_ :: Parser Value
array_ = {-# SCC "array_" #-} do
skipSpace
vals <- ((value <* skipSpace) `sepBy` (char ',' *> skipSpace)) <* char ']'
- return . Array $ Vector.fromList vals
+ return $! Array $! Vector.fromList vals

-- | Parse any JSON value. Use 'json' in preference to this function
-- if you are parsing data from an untrusted source.
value :: Parser Value
-value = most <|> (Number <$> number)
+value = most <|> fallback
where
+ fallback = do
+ n <- number
+ return $! Number $! n
+
most = do
c <- satisfy (`B8.elem` "{[\"ftn")
case c of
```

This leads to the following benchmark results:

```
-- lazy aeson-0.3.2.12 w/ 'evaluate $ rnf r' benchmark
$ ./AesonParse 700 json-data/jp100.json
json-data/jp100.json:
700 good, 3.459478s
202 per second

-- strictified aeson-0.3.2.12 w/ 'evaluate $ rnf r' benchmark
$ ./AesonParse 700 json-data/jp100.json
json-data/jp100.json:
700 good, 3.176462s
220 per second

-- Python 2.7
$ ./parse.py 700 json-data/jp100.json
json-data/jp100.json:
1.49782395363
```