Repository: hylang/hy

Issue:
A statement pulled out of the condition of a `while` loop is only evaluated once
Thinking about #1342 made me guess that the following would do the wrong thing, and indeed, it does:

    $ echo '(setv x 3) (while (do (print "hi") x) (print x) (-= x 1))' | hy2py
    x = 3
    print('hi')
    while x:
        print(x)
        x -= 1

The `print('hi')` only happens once instead of every time the `while` condition is checked, because it's pulled out to before the `while`.

The most straightforward fix is to make a `while` loop with a complex condition compile to something like:

    x = 3
    while True:
        # Condition
        print('hi')
        _hy_anon_var_1 = x
        if not _hy_anon_var_1:
            break
        # Body
        print(x)
        x -= 1

(In practice, `Result.rename` would remove the use of `_hy_anon_var_1` and just write `if not x:`.)

The problem with this method is that an `else` clause won't be executed when the loop terminates normally, because `break` skips the `else`. This doesn't actually matter right now because we haven't yet implemented `else` for `while`. But a method that's robust to this future added feature is:

    x = 3
    # Condition
    print('hi')
    _hy_anon_var_1 = x
    while _hy_anon_var_1:
        # Body
        print(x)
        x -= 1
        # Condition, repeated
        print('hi')
        _hy_anon_var_1 = x

Note that it will then be unwise to use `break` or `continue` inside the condition, because the condition executes once outside the `while` and subsequently inside it.



Discussion:
How *should* a `break`/`continue` work in the condition? I think these statements are only supposed to be valid in a loop *body*. We'd get a `SyntaxError: 'break' outside loop` before we even got to the inner one. And this is fine.

The problem with the second option is what happens with nested loops. If the `while` with a `break` in the condition is nested inside another loop, then it will break the outer loop on the first iteration, but the inner loop after that, which is weird and inconsistent. Similarly with `continue`.

The first option only puts the condition inside the body, so it will only break the *inner* loop, which is still weird, but at least it's consistent.

We could probably disallow `(break)` or `(continue)` inside a `while` condition in Hy's compiler altogether. I don't see a use for it anyway. I'm not sure how easily we could detect this though. We'd need a context stack, since another `while` should be allowed in the condition, and there's nothing wrong with putting a `break` in that.

Maybe there's some other way to compile this that gives us sensible `break`/`continue` without corrupting `else`.
A third option.
```Python
x = 3
_hy_anon_var_1 = True
while _hy_anon_var_1:
    # Condition
    print('hi')
    _hy_anon_var_2 = x
    if _hy_anon_var_2:
        # Body
        print(x)
        x -= 1
    else:
        _hy_anon_var_1 = False
```
This is like the first option, but we don't add our own `break`, so the `else` clause for the `while` should work properly. The condition also always happens inside the loop, so `break`/`continue` are consistent.

[Edit: we don't need two anon vars]
```Python
x = 3
_hy_anon_var_1 = True
while _hy_anon_var_1:
    # Condition
    print('hi')
    _hy_anon_var_1 = x
    if _hy_anon_var_1:
        # Body
        print(x)
        x -= 1
```
> If the `while` with a `break` in the condition is nested inside another loop, then it will break the outer loop on the first iteration, but the outer loop after that, which is weird and inconsistent.

Right, that's what I was thinking.

I'm not too worried about this in any case because putting a loop-control statement in a loop condition is a weird thing to do in the first place. "Doctor, it hurts when I do this", etc.

Here's another version I thought of that's similar to your version:

    x = 3
    _hy_anon_var_1 = True
    while _hy_anon_var_1:
        # Condition
        print('hi')
        _hy_anon_var_1 = bool(x)  
        if _hy_anon_var_1:
            # Body
            print(x)
            x -= 1

The `bool` is so the anon var stays the same even if the truth value of the object referred to by `x` changes (e.g., if `x` is a list and it's emptied).
Oh, jinx!
Haha. I realized I didn't need two anon vars so I edited it to simplify. I put the original version back for posterity.
Good point about using `bool`. A mutation in the body could prematurely interrupt the loop, without executing the condition clause again. But since Python builtins can be shadowed in a module, I'd prefer not to rely on them in the compiler when they aren't explicitly asked for, if possible. If we *have* to use a builtin, it should compile like--
```Python
from builtins import bool as _hy_anon_var_2  # from __builtin__ in Python2
_hy_anon_var_1 = _hy_anon_var_2(x)
```
But in this case, we could avoid it altogether like this:
```Python
x = 3
_hy_anon_var_1 = True
while _hy_anon_var_1:
    # Condition
    print('hi')
    _hy_anon_var_1 = x
    if _hy_anon_var_1:
        _hy_anon_var_1 = True
        # Body
        print(x)
        x -= 1
```
True. You could also write `not not x`.
Hi, first: great job here with Hy. I'm looking forward for version 1.0.

About this issue, why not creating a local function?

```
x=3
def _hy_anon_def_1():
  print('hi')
  return x
while _hy_anon_def_1():
  print(x)
  x -= 1
```

Does that mess up with the AST too much?
We don't create anonymous functions when we don't have to because each function creates a new scope, meaning that `setv` won't set the variable that the user expects. Also, now that we have `return`, we need to make sure it returns from the right function.

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -52,5 +52,5 @@
     hy_descriptive_version = hy_descriptive_version)
 
 intersphinx_mapping = dict(
-    py2 = ('https://docs.python.org/2', None),
-    py  = ('https://docs.python.org/3', None))
+    py2 = ('https://docs.python.org/2/', None),
+    py  = ('https://docs.python.org/3/', None))
diff --git a/hy/compiler.py b/hy/compiler.py
--- a/hy/compiler.py
+++ b/hy/compiler.py
@@ -1861,12 +1861,33 @@ def compile_for_expression(self, expression):
     @checkargs(min=2)
     def compile_while_expression(self, expr):
         expr.pop(0)  # "while"
-        ret = self.compile(expr.pop(0))
+        cond = expr.pop(0)
+        cond_compiled = self.compile(cond)
 
-        orel = Result()
-        # (while cond body (else …))
+        else_expr = None
         if ends_with_else(expr):
             else_expr = expr.pop()
+
+        if cond_compiled.stmts:
+            # We need to ensure the statements for the condition are
+            # executed on every iteration. Rewrite the loop to use a
+            # single anonymous variable as the condition.
+            def e(*x): return HyExpression(x)
+            s = HySymbol
+            cond_var = s(self.get_anon_var())
+            return self.compile(e(
+                s('do'),
+                e(s('setv'), cond_var, 1),
+                e(s('while'), cond_var,
+                  # Cast the condition to a bool in case it's mutable and
+                  # changes its truth value, but use (not (not ...)) instead of
+                  # `bool` in case `bool` has been redefined.
+                  e(s('setv'), cond_var, e(s('not'), e(s('not'), cond))),
+                  e(s('if*'), cond_var, e(s('do'), *expr)),
+                  *([else_expr] if else_expr is not None else []))).replace(expr))  # noqa
+
+        orel = Result()
+        if else_expr is not None:
             for else_body in else_expr[1:]:
                 orel += self.compile(else_body)
                 orel += orel.expr_as_stmt()
@@ -1874,11 +1895,9 @@ def compile_while_expression(self, expr):
         body = self._compile_branch(expr)
         body += body.expr_as_stmt()
 
-        ret += asty.While(expr,
-                          test=ret.force_expr,
-                          body=body.stmts,
-                          orelse=orel.stmts)
-
+        ret = cond_compiled + asty.While(
+            expr, test=cond_compiled.force_expr,
+            body=body.stmts, orelse=orel.stmts)
         ret.contains_yield = body.contains_yield
 
         return ret


Test patch:
diff --git a/tests/native_tests/language.hy b/tests/native_tests/language.hy
--- a/tests/native_tests/language.hy
+++ b/tests/native_tests/language.hy
@@ -346,6 +346,68 @@
   (assert (= a [2 "e"])))
 
 
+(defn test-while-multistatement-condition []
+
+  ; The condition should be executed every iteration, before the body.
+  ; `else` should be executed last.
+  (setv s "")
+  (setv x 2)
+  (while (do (+= s "a") x)
+    (+= s "b")
+    (-= x 1)
+    (else
+      (+= s "z")))
+  (assert (= s "ababaz"))
+
+  ; `else` should still be skipped after `break`.
+  (setv s "")
+  (setv x 2)
+  (while (do (+= s "a") x)
+    (+= s "b")
+    (-= x 1)
+    (when (= x 0)
+      (break))
+    (else
+      (+= s "z")))
+  (assert (= s "abab"))
+
+  ; `continue` should jump to the condition.
+  (setv s "")
+  (setv x 2)
+  (setv continued? False)
+  (while (do (+= s "a") x)
+    (+= s "b")
+    (when (and (= x 1) (not continued?))
+      (+= s "c")
+      (setv continued? True)
+      (continue))
+    (-= x 1)
+    (else
+      (+= s "z")))
+  (assert (= s "ababcabaz"))
+
+  ; `break` in a condition applies to the `while`, not an outer loop.
+  (setv s "")
+  (for [x "123"]
+    (+= s x)
+    (setv y 0)
+    (while (do (when (and (= x "2") (= y 1)) (break)) (< y 3))
+      (+= s "y")
+      (+= y 1)))
+  (assert (= s "1yyy2y3yyy"))
+
+  ; The condition is still tested appropriately if its last variable
+  ; is set to a false value in the loop body.
+  (setv out [])
+  (setv x 0)
+  (setv a [1 1])
+  (while (do (.append out 2) (setv x (and a (.pop a))) x)
+    (setv x 0)
+    (.append out x))
+  (assert (= out [2 0 2 0 2]))
+  (assert (is x a)))
+
+
 (defn test-branching []
   "NATIVE: test if branching"
   (if True