feeley/gambit — pull request #69 by matthastie (2014-01-12 11:54:14+00:00)
Title: Provides DSSSL support in syntax-case

I think there are three interesting implementation details worth highlighting, as I suspect that they might apply to the general implementation of DSSSL in the context of any syntax expander. Points 2 & 3 certainly weren't obvious to my untrained eye when I commenced the work, and so I hope this discussion may enlighten those who may want DSSSL in other syntax expander implementations.

1. Implementation of formals that include specifically `#!optional` and `#!rest` are simple - the variables are alpha-converted, and the DSSSL directives are passed-through to Gambit, which provides native DSSSL support. For example,

```
> ((lambda (a #!optional (b 3) #!rest r) `(,a ,b ,r)) 1 4 5 6)
((lambda (%%a0 #!optional (%%b1 '3) #!rest %%r2) (list %%a0 %%b1 %%r2)) 1 4 5 6)
(1 4 (5 6))
>
```
In this case, `a` `b` and `r` are alpha-converted to `%%a0` `%%b1` and `%%r2` respectively. The main alteration psyntax needed to support this use case was the filtering of `#!optional` and `#!rest` from variable lists to avoid triggering lambda syntax errors. While this would appear a simple win, further complexity lurked in the implementation of default and keyword parameters.

2. Default parameters are expressions! Thus it is possible for the mad Schemer to do things like this:

```
(let ((a 1))
((lambda (#!optional (b (begin 'gratuitous-side-effect-ftw (+ 1 a)))) b)))
```

In this case, the default parameter of `b` is expressed in terms of `a` inside a begin statement. In psyntax, `a` is alpha-converted in the let binding, and so each default expression also requires alpha-conversion. The output for this example thus looks like this:

```
((lambda (%%a5)
((lambda (#!optional (%%b6 (begin 'gratuitous-side-effect-ftw (+ 1 %%a5))))
%%b6))) 1)
2
```

3. Keyword parameters expose environment to the application. When one writes, for example:

```
(define foo
(lambda (#!keyword a) a))
```
The binding `a` becomes exposed in the environment of the application as a keyword, so that one may write

```
(foo a: 1)
```

This 'injection of environment' is problematic for any syntax expander that alpha-converts symbols; the keyword arguments also need alpha-conversion for correctness. This means that all applications of the lambda definition must be cognizant of the converted symbol names. For `foo` above, alpha-conversion may yield:

```
(define %%foo23
(lambda (#!keyword %%a24) %%a24))

(foo a: 1) ;; is now wrong, and now needs to be
(foo %%a24: 1)
```

When choosing the implementation provided, I contemplated three possible solutions:

a) Avoid alpha-conversion of keyword parameters. For example, emit something like:

```
(define %%foo23
(lambda (#!keyword a) a))
```

While this solution is simple, I did fear that it would break syntax hygiene when keywords were used. I thus avoided it.

b) Extend the environment of applications to include the keyword variable alpha-conversions. I personally think this is the optimal choice for performance and correctness purposes. However, I confess that reading the psyntax codebase to make this change made my eyes bleed, I couldn't identify simple strategic edits that could achieve such a solution. Furthermore, I identified that unit compilation of any such solution would be difficult: it necessitates publishing the keyword environment for all keyworded lambda in a visit file, so that separate compilation units can be made aware of the alpha-conversions ... quite a nasty problem that I did not wish to solve. One additional detail to contemplate is the need to avoid alpha-conversion of keyword parameters in applications of native Gambit functions. With all this to consider, I opted for solution C.

c) Place the application environment within a non-alpha-converted closure within the emitted lambda construct. This yields a simple solution that preserves hygiene at the cost of performance. The emitted code looks like this (speed freaks should overt eyes to prevent foaming at mouth):

```
> ((lambda (#!key a) a) a: 1)
((lambda %%dsssl-args10
(receive (%%a9) (apply (lambda (#!key a) (values a)) %%dsssl-args10)
%%a9)) a: 1)
1
>
```

Thus, in this solution all keyword parameters are alpha converted, and the original lambda expression is placed in the body of the receive block. I believe hygiene is preserved due to the evaluation of keywords in the apply, and client code to the syntax expander cannot dereference the generated `%%dsssl-args10`.

I've included some unit tests for this work in the pull request, with intent to provide reasonable coverage for DSSSL argument variations up to this level of complexity, with both `#!rest` and `( . rest)` variants covered:

```
> ((lambda (x #!optional o #!key a . r) `(,x ,o ,a ,r)) 4 1 a: 2 3 5 6)
((lambda %%dssl-args15
(receive (%%x14 %%o13 %%a12 %%r11)
(apply (lambda (x #!optional o #!key a . r) (values x o a r))
%%dssl-args15)
(list %%x14 %%o13 %%a12 %%r11)))
4 1 a: 2 3 5 6)
(4 1 2 (3 5 6))
>
```

Kind regards,
Matt.