JuliaStats/DataFrames.jl — pull request #472 by johnmyleswhite (2014-01-11 17:33:11+00:00)
Title: [RFC] Establishing a Julian precedent for translating R functions that use delayed evaluation

This small PR implements a basic `@select` macro. This macro is just a demo, which I'm using to start a conversation about how I'd like Julia to approach problems for which R would make use of delayed evaluation.

The core premises behind this new approach are simple:

**Premise 1**: Many uses of delayed evaluation in R can be conceptualized as the extension of the caller's scope to include new variables that correspond to the column names of a DataFrame. This can only be done at run-time when the names of the columns are known, which is why we've been imitating R operations by passing expressions to DataFrames. But that approach has proven hard to work with, because it means that many parts of the caller's scope are difficult to access without careful escaping operations. It also makes code a little ugly, because one has to write things like `df[:(A > $x), :]`.

**Premise 2**: Julia should not provide delayed evaluation until we know that it is unavoidable. Instead of doing computations that augment the caller's scope, we should provide a tool for mixing column names with the caller's scope that can be handled entirely at macro compile-time. This requires that the user state at code-writing time which variables are column names. *I would argue that this added burden is a good thing, because it makes the magic we're about to perform more explicit and systematic.*

**Premise 3**: We can have users distinguish the caller's scope from the column names of a DataFrame by asking them to write explicit symbols, which are turned into quoted symbols by the parser. Thus the caller's `x` is written as `x`, whereas the column name `x` is written as `:x`.

The examples below demonstrate this approach and show that it substantially simplifies indexing DataFrames using complex expressions:

using DataArrays
using DataFrames

df = DataFrame(A = 1:3, B = [2, 1, 2])
x = [2, 1, 0]

@select df :A .> 1
@select df :B .> 1
@select df :A .> x
@select df :B .> x
@select df :A .> :B
@select df sin(:A) .> cos(:B)
@select df cos(:A) .> sin(:B)
@select df cos(x) .> sin(:B) + 1
@select df sin(x) .> sin(:B) + 1
@select df sin(:A) .> sin(:B) + 1
@select df sin(:A) .> sin(:B)
@select df sin(:A) .> exp(cos(:B) - 1)

Importantly, the indirection introduced by the `@select` macro will allow us to gradually reimplement these operations in more efficient ways that exploit indices in DataFrames. It will also allow us to implement these operations in a way that, like LINQ, would apply to both DataFrames and databases.