Netflix/PigPen — pull request #39 by mbossenbroek (2014-06-17 21:48:20+00:00)
Title: Fix local UDF memoization

Third try is the charm :)

Attempt 1 memoized every UDF creation, but this led to a sticky REPL situation. For example: define function `foo`, use it in a `pig/map` command, and you can never redefine it because the memoization looks for the symbol `foo` - not the contents.

Attempt 2 didn't do any memoization, but this led to poor performance (every piece of data re-eval'd the UDF code) and eventually ran out of permgen.

Hopefully, attempt 3 will be just right. I added a udf-scope symbol that is passed to the memoized function that creates the UDF and evals user code. This is re-created every time we evaluate a pigpen expression. Unfortunately, I couldn't thread it through graph->local as that would break a lot of existing code that doesn't live here, so I created a dynamic var instead. This should be cleaned up when other breaking changes are made to this code.