Repository: hylang/hy

Issue:
`with` forms cannot parse multiple non-named with-exprs
Currently, `with` parses like so (simplification mine):
```
=> (with [a b c d e] ...)
with b as a:
  with d as c:
    with e:
      ...
```

As a result, there is no way of writing multiple non-named forms, nor is there any way to have a non-named form anywhere other than the end.

I propose the following instead, with a set of outer brackets for the with-exprs and inner brackets for any named with-exprs:

```
=> (with [[a b] c d [e f]] ...)
with b as a, c, d, f as e:
  ...
```


Discussion:
I guess that we should also get rid of `with*` since Python's `with` can now accept multiple clauses.
All that sounds sensible to me. I'm a bit embarrassed I never noticed that deficiency in `with`, but I guess I never use more than one context manager at a time, anyway.
Given that most of the time you only want to use a single context expression, it seems quite verbose to force users to add another pair of brackets in the most common case.

Would it not make sense to just write `_` when you do want lots of non-named forms?
```hy
(with [_ expr1 _ expr2] ...)
```
I admit that I don't like having to use double brackets in one of the most common cases, either. I think that was required in an earlier version of Hy, in fact. Using `_` as a placeholder variable name is dangerous in Hy because it can shadow the shadow subtraction operator; even if this was a special case in which `_` was recognized by the compiler and no binding variable was actually emitted, we don't want to encourage the habit. Any other ideas for the syntax?
> Using `_` as a placeholder variable name is dangerous in Hy because it can shadow the shadow subtraction operator;

This seems pretty bad. I didn't realise that this could actually happen. I came up with a small example:

``` hy
=> (with [_ (open "file.txt")] (reduce - [1 2 3]))
TypeError: '_io.TextIOWrapper' object is not callable
```

This seems like a bigger issue to me given how prevalent using `_` is in Python and many other languages. Would it not make sense to have the compiler treat `_` as special everywhere? Make it so that `_` is always compiled to `_` and `-` to `-`.

> Any other ideas for the syntax?

If we can't use and underscore, the only thing I can suggest is using some other variable name that you wouldn't care about.
This is probably too goofy, but I thought of allowing a literal `0` in the place of a variable, which would be interpreted by the compiler as a request to not bind the context manager. That is, `(with [a b  0 c  d e] …)` would be compiled to `with b as a, c, e as d: …`.
It would make the parsing a little more complicated, but we could have `with` accept ~~two~~ three different patterns:

```hy
(with [single-form] body)
(with [even number of forms] body)
(with [[explicitly grouped] [forms]] body)
```

The downside is that a with statement with many single forms now looks pretty gross:
```hy
(with [[many] [single] [forms]] ...)
```
But this is relatively rare (I think?) and could also be sugared over with a new macro.
This is similar to what I was going to propose. the only difference being form 2 would only allow one ctx manager and its binding:
```clojure
(with [single-form] body)
(with [sym bound-form] body)
(with [[explicitly grouped] [forms]] body)
```
> ```hy
> (with [single-form] body)
> (with [even number of forms] body)
> (with [[explicitly grouped] [forms]] body)
> ```
This seems pretty reasonable.


Resolving patch:
diff --git a/hy/compiler.py b/hy/compiler.py
--- a/hy/compiler.py
+++ b/hy/compiler.py
@@ -896,36 +896,39 @@ def compile_decorate_expression(self, expr, name, args):
         fn.stmts[-1].decorator_list = decs + fn.stmts[-1].decorator_list
         return ret + fn
 
-    @special(["with*", "with/a*"],
-             [brackets(FORM, maybe(FORM)), many(FORM)])
+    @special(["with", "with/a"],
+             [(brackets(times(1, Inf, FORM + FORM)))
+              | brackets((FORM >> (lambda x: [(HySymbol('_'), x)]))),
+              many(FORM)])
     def compile_with_expression(self, expr, root, args, body):
-        thing, ctx = (None, args[0]) if args[1] is None else args
-        if thing is not None:
-            thing = self._storeize(thing, self.compile(thing))
-        ctx = self.compile(ctx)
-
         body = self._compile_branch(body)
 
         # Store the result of the body in a tempvar
-        var = self.get_anon_var()
-        name = asty.Name(expr, id=mangle(var), ctx=ast.Store())
+        temp_var = self.get_anon_var()
+        name = asty.Name(expr, id=mangle(temp_var), ctx=ast.Store())
         body += asty.Assign(expr, targets=[name], value=body.force_expr)
         # Initialize the tempvar to None in case the `with` exits
         # early with an exception.
         initial_assign = asty.Assign(
             expr, targets=[name], value=asty.Constant(expr, value=None))
 
-        node = asty.With if root == "with*" else asty.AsyncWith
-        the_with = node(expr,
-                        context_expr=ctx.force_expr,
-                        optional_vars=thing,
-                        body=body.stmts,
-                        items=[ast.withitem(context_expr=ctx.force_expr,
-                                            optional_vars=thing)])
+        ret = Result(stmts=[initial_assign])
+        items = []
+        for variable, ctx in args[0]:
+            ctx = self.compile(ctx)
+            ret += ctx
+            variable = (None
+                if isinstance(variable, HySymbol) and variable == HySymbol('_')
+                else self._storeize(variable, self.compile(variable)))
+            items.append(asty.withitem(expr,
+                                       context_expr=ctx.force_expr,
+                                       optional_vars=variable))
+
+        node = asty.With if root == "with" else asty.AsyncWith
+        ret += node(expr, body=body.stmts, items=items)
 
-        ret = Result(stmts=[initial_assign]) + ctx + the_with
         # And make our expression context our temp variable
-        expr_name = asty.Name(expr, id=mangle(var), ctx=ast.Load())
+        expr_name = asty.Name(expr, id=mangle(temp_var), ctx=ast.Load())
 
         ret += Result(expr=expr_name)
         # We don't give the Result any temp_vars because we don't want


Test patch:
diff --git a/tests/compilers/test_ast.py b/tests/compilers/test_ast.py
--- a/tests/compilers/test_ast.py
+++ b/tests/compilers/test_ast.py
@@ -359,9 +359,9 @@ def test_ast_bad_assoc():
 
 def test_ast_bad_with():
     "Make sure AST can't compile invalid with"
-    cant_compile("(with*)")
-    cant_compile("(with* [])")
-    cant_compile("(with* [] (pass))")
+    cant_compile("(with)")
+    cant_compile("(with [])")
+    cant_compile("(with [] (pass))")
 
 
 def test_ast_valid_while():
diff --git a/tests/native_tests/contrib/walk.hy b/tests/native_tests/contrib/walk.hy
--- a/tests/native_tests/contrib/walk.hy
+++ b/tests/native_tests/contrib/walk.hy
@@ -58,26 +58,14 @@
   ;; make sure a macro from the current module works
   (assert (= (macroexpand-all '(foo-walk))
              42))
-  (assert (= (macroexpand-all '(with [a 1]))
-             '(with* [a 1])))
+  (assert (= (macroexpand-all '(-> 1 a))
+             '(a 1)))
   ;; macros within f-strings should also be expanded
   ;; related to https://github.com/hylang/hy/issues/1843
   (assert (= (macroexpand-all 'f"{(foo-walk)}")
              'f"{42}"))
-  (assert (= (macroexpand-all 'f"{(with [a 1])}")
-             'f"{(with* [a 1])}"))
-  (assert (= (macroexpand-all '(with [a 1 b 2 c 3] (for [d c] foo)))
-             '(with* [a 1] (with* [b 2] (with* [c 3] (for [d c] foo))))))
-  (assert (= (macroexpand-all '(with [a 1]
-                                 '(with [b 2])
-                                 `(with [c 3]
-                                    ~(with [d 4])
-                                    ~@[(with [e 5])])))
-             '(with* [a 1]
-                '(with [b 2])
-                `(with [c 3]
-                   ~(with* [d 4])
-                   ~@[(with* [e 5])]))))
+  (assert (= (macroexpand-all 'f"{(-> 1 a)}")
+             'f"{(a 1)}"))
 
   (defmacro require-macro []
     `(do
diff --git a/tests/native_tests/with_test.hy b/tests/native_tests/with_test.hy
--- a/tests/native_tests/with_test.hy
+++ b/tests/native_tests/with_test.hy
@@ -2,6 +2,8 @@
 ;; This file is part of Hy, which is free software licensed under the Expat
 ;; license. See the LICENSE.
 
+(import pytest)
+
 (defclass WithTest [object]
   (defn __init__ [self val]
     (setv self.val val)
@@ -44,6 +46,15 @@
         (assert (= t2 2))
         (assert (= t3 3))))
 
+(defn test-unnamed-context-with []
+  "`_` get compiled to unnamed context"
+  (with [_ (WithTest 1)
+         [b d] (WithTest (range 2 5 2))
+         _ (WithTest 3)]
+    (assert (= [b d] [2 4]))
+    (with [(pytest.raises NameError)]
+      _)))
+
 (defclass SuppressZDE [object]
   (defn __enter__ [self])
   (defn __exit__ [self exc-type exc-value traceback]