Repository: hylang/hy

Issue:
Return expression
Heresy?  But maybe we should provide:

``` clojure
(defn try-this [arg]
 (if (= arg "immediately return")
   ;; bail out early
   (return arg))
 ;; a whole bunch of code in-between
 ;; [...]
 ;; The end-of-function return
 (something-something arg blah))
```



Discussion:
This could be implemented with a decorator and an Exception, like the tail
call. I.e. return is a function that raises a "HyReturnExcption" and the
wrapped function just catches this and returns that value.
On þri., 30. des. 2014 at 18:31 Christopher Allan Webber <
notifications@github.com> wrote:

> Heresy? But maybe we should provide:
> 
> (defn try-this [arg](if %28= arg)
>    ;; bail out early
>    (return arg))
>  ;; a whole bunch of code in-between
>  ;; [...]
>  ;; The end-of-function return
>  (something-something arg blah))
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/hylang/hy/issues/739.

@Tritlo you can implement this using the return AST form. Not really hard, but @paultag have been against explicit return statements.

It would indeed encourage a non functional style, but I still think this
should be an option. Maybe a **future** import, like in #728
On þri., 30. des. 2014 at 20:08 Morten Linderud notifications@github.com
wrote:

> @Tritlo https://github.com/Tritlo you can implement this using the
> return AST form. Not really hard, but @paultag
> https://github.com/paultag have been against explicit return statements.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/hylang/hy/issues/739#issuecomment-68392830.

I think there's no need for a **future** import... why not just include it as an option, then strongly encourage using the other pattern instead?

I do think there are points in code where not using a (return) is hard, especially without abusing exceptions or resulting in nested ifs that aren't really ideal.  They're comparatively rare, but still.

Some other lisps have ways of getting around this; for example, in Common Lisp I have blocks, so I can do

``` lisp
(defun foo ()
  (block 
      got-result
    1
    2
    (return-from got-result 3)
    4))
```

Sometimes, especially in Python code, bailing out early is nice.  We should encourage other forms in Hy, but why prevent users from using this function?

I might understand more if it breaks other complicated-scoping code, but is that really the reason?

Actually, it turns out common lisp sets up the function as a block, so:

```
(defun foo ()
  1
  2
  (return-from foo 3)
  4)
```

Can I also vote for the return expression? I think Hy should be able to do anything Python can. I should be able to translate any Python code I see into near-equivalent Hy. Automatically even, with some kind of py2hy (de)compiler. This would be really hard without return. You would have to rewrite any Python code with multiple exits.

I'm fine with having an implicit return at the tail position. I'd prefer not to have to spell it out if I don't have to.

The return statement is Rossum's [recommended way](https://mail.python.org/pipermail/python-3000/2007-July/008663.html) of breaking out of nested loops. This isn't possible with only an implicit return.

If I'm writing the code myself from scratch, I'd prefer using higher-order functions over explicit nested loops anyway, but that doesn't mean I want to translate someone's working algorithm into that style just to insert some Hy macros.

@hylang/core opinions? Might be a good idea for the cleanup

I'm -1 on return, but let's open a bug and discuss. Basically, I'm worried
we'll wind up with a Ruby-style mess, and having return values be confusing
to grok.

On Fri, Jul 24, 2015 at 11:02 AM, Morten Linderud notifications@github.com
wrote:

> @hylang/core https://github.com/orgs/hylang/teams/core opinions? Might
> be a good idea for the cleanup
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/hylang/hy/issues/739#issuecomment-124551856.

## 

:wq

I'm also -1 on return. Mixing explicit and implicit returns would be confusing, in my opinion.

I really don't think explicit returns are any more confusing than `break` or `continue` inside loops, or the whole exception system. You aren't proposing we cut those out too? You'll get used to it. As for mixing explicit and implicit, well, Ruby might be a bad example, but I'll point out that Rust also does it this way. Even Python has an implicit return in `lambda` expressions, but explicit `return` in function `def`s.

I feel pretty strongly that Hy functions really ought to have multiple exits, because this is something Python already does, and "We're still Python(TM).".

After re-reading my last comment I had another idea. While I have absolutely no problem with mixing implicit and explicit returns, if that's the main sticking point preventing Hy from having multiple exit capability then, then why not offer both, but cleanly separate them to avoid any mixing?

We could have `defn`/`fn` require an explicit `return` even in the tail position, while `lambda` disallows any `return` and has the implicit return at the tail. **Python users will not be surprised by this behavior.**

Unlike Python, the `lambda` in Hy already has the implicit progn feature, or as near as we can get, I guess, so it's still OK to use "statements" in a Hy lambda. We would only have to change it to complain if there's a `return` form.

One complaint I would have with this arrangement is that if we ideally want to prefer and encourage the implicit forms in Hy code, then between `fn` and `lambda`, the preferred form should not have the longer name. This is easy to fix; we just need a shorter symbol for `lambda`. The possibilities `la`, `lam`, `/\`, `\`, `.\` and `,\` come to mind,

I'm also -1 on return. I really like building code where last value of function is returned, without having to have `return` there. So far I haven't had a case where I had needed to return a value from middle of a function. But, I like to chop things into relatively tiny pieces anyway. All of this is of course just a personal preference.

I think a neat idea would be to have metadata inside functions. Clojure
does this, and i think its an interesting appraoch to problems like
these.

``` clojure
(defn function [a b c]
  "Function with explicit return"
  {:return True}
  (return [a b c]))
```

Thoughts?

On Fri, Jul 24, 2015 at 01:49:09PM -0700, Tuukka Turto wrote:

> I'm also -1 on return. I really like building code where last value of function is returned, without having to have `return` there. So far I haven't had a case where I had needed to return a value from middle of a function. But, I like to chop things into relatively tiny pieces anyway. All of this is of course just a personal preference.
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/hylang/hy/issues/739#issuecomment-124718232

Metadata sounds like a interesting approach, if we indeed want to have explicit return statements. We wouldn't have to have multiple different forms for almost same thing. Just have to make sure, that is somebody for some funky reason wants to return a dict `{:return True}` it is possible. I still think that explicit return statement is something that the language doesn't really need though.

The line could be done before a comment or something. It's a suggestion, syntax isn't that important yet.

> We wouldn't have to have multiple different forms for almost same thing.

I am generally opposed to having multiple forms for almost the same thing, at least in the core. Adding new forms is easy with macros. Subtracting cruft, not so much. But I do think it's okay in this case because Python already has `lambda` even though `def` does almost the same thing. Rather than adding complications, it would make Hy fit Python even better. And after all, We're still Python.

> I really like building code where last value of function is returned, without having to have return there.

Me too, but in neither of my proposals would you lose this ability.

> So far I haven't had a case where I had needed to return a value from middle of a function.

One can always work around such cases, but that doesn't mean we should have to. Writing new Hy from scratch is one thing, but this commonly comes up when translating existing Python into Hy. For most of Python this is easy, but it's a pain to rewrite the flow so there's only one exit. Python has no goto, and no labeled break--even though it's helpful for nested loops. I think the main argument for not adding this to Python is that you can break out if any amount of nesting with a return. So that's how it's done. But in Hy I can't even do that. The only thing left in Hy to penetrate deep nesting is to raise an exception.

> I think a neat idea would be to have metadata inside functions. ...

If that's the only way I get multiple exits, I can live with that, but I still like either of my proposals better than this. Metadata has to live somewhere. More runtime overhead I suppose. Python already allows getattr/setattr for functions, so you can store metadata that way. It seems like a better, more Pythonic approach than tacking on a new Clojure-style metadata system. 

My main objection is not ascetic. Two ways to do something is something I'd
prefer to avoid :)
On Feb 25, 2016 5:57 PM, "ez47" notifications@github.com wrote:

> I'm also -1 on return. I really like building code where last value of
> function is returned, without having to have return there.
> 
> kill yourself, seriously what kind of argument is that?
> 
> return function should be included not only for the compatibility between
> python but even beyond that, a truely functional way of programming is the
> specification pattern where you SPECIFY both the INPUT and the OUTPUT .
> with which you can chain and compose arbitrary pieces of logic.
> 
> instead of this
> '''
> (defn specify clause &rest work
> (eval exePointer))
> 
> (specify (not form) (do_something) '("no form"))
> '''
> which doesnt even work.
> you could have something like
> 
> '''
> (specify (not form) (do_something) '(return "no form"))
> '''
> 
> Languages shouldnt be treated like some artsy past time, 1 return per
> function is a high level human constraint, not a logical one.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/hylang/hy/issues/739#issuecomment-189020861.

> kill yourself, seriously what kind of argument is that? 

Yeah no. That's just horrible. 
Read https://github.com/hylang/hy/blob/master/CONTRIBUTING.rst and try again if you want to voice your opinions on issues on this project.

@ez47 your attitude is entirely unwelcome and that you're not constructively adding to the conversation.

> kill yourself

Is not appropriate.

I also feel like these comments are not adding any signal to this bug.

Locking this issue. Folks can prod me about this but I'm not enjoying the way this went.

Can this be closed now? Evidently, this did _not_ go well...

We may want to reevaluate whether an explicit `return` is useful. I can do without it, but in the time between my previous diapproval and now, I became less... opinionated. So +0, I guess? :)

@algernon Same here. I'm indifferent either way. In Lisp languages, explicit returna are rarely used anyway, since you break up problems into smaller functions.

Thus spake @paultag:

> My main objection is not ascetic. Two ways to do something is something I'd
> prefer to avoid :)

Or as stated in the Zen of Python:

> There should be one-- and preferably only one --obvious way to do it.

If you're willing to break this rule, you get Perl. I get it. 

We should be very reluctant to add feature bloat, and should remove as much cruft as we can given this rare opportunity to break API. But don't forget, Hy is a Lisp. There's a lesson we've learned from the history of Scheme: the users will build what they need (using _macros_ if necessary!) when the language doesn't have it--_in a myriad of incompatible ways_! There should be one-- and preferably only one --obvious way to do it. Zero is not one either.

I think we should be _very cautious_ about deciding to diverge from Python itself. We're dependent on Python. Python does add features from time to time--features that may depend on existing Python syntax, like `return`.

If Python gets too far past us, Hy becomes more and more obsolete as new software is written that depends on these features that Hy can't use. Why use Hy at all when we already have Common Lisp, Scheme, and Clojure? The main reason: Hy is a Lisp that can work with Python's libraries as easily as its own. Hy must be able to do anything Python can to remain relevant. Anything Python can do that Hy can't do is a reason not to use it. Sometimes it only takes one good reason. Hy doesn't have a big enough ecosystem to compete on its own.

Those are the principles. I'm less certain about the solution, but I'm open to ideas.

My best idea so far: Add a `return` form. Keep `lambda`'s implicit return as-is, and disallow `return` directly inside a `lambda`, even when Hy translates it to multiple statements. Add a `deflambda` with the same rules. Require an explicit return in `fn`/`defn`, and allow Python's `None` return to happen if an explicit return isn't provided.
- Python's `lambda` already has an implicit return, so this is familiar to Python users.
- We don't have to remember to return `nil` when defining a method like `__init__`, or rely on fragile automagic to do it for us.
- generators don't break in older Python versions when Hy decides to automatically insert a `return` in a function that already contains `yield`.

It's a little troubling that the special-case syntax `fn` (requiring `return`) is shorter than the common case `lambda`. There's always more than one way to do it, but we can make the preferred method more **obvious**. For example, we could rename `lambda` to `^` and `defn` to `define-function` or something like that.

@gilch Huh...That's an interesting idea.

I'm personally torn. On one hand, implicit returns are awesome. On the _other_ hand, they do have problems like you said.

One idea could be to make returns explicit inside a generator and make a clear distinction via `defgn`/`gn` like you said in another issue. Problem? It still doesn't account for the whole `__init__`, and defining context managers via decorators would be a pain in the neck.

Implicit returns are great in my opinion and most of the time explicit `return` would just add noise:

``` hy
(defn add [a b]
  (+ a b))

vs.

(defn add [a b]
  (return (+ a b)))
```

However, there are cases where explicit return is really useful (for example, returning a value from inside of a `while`) and not having access to one means that you have to work around that with other mechanisms (`setv` to store result, `break` to exit loop and then bare result so it gets returned).

If we add `return`, can we do that in a way that doesn't break each and every `defn` and `fn` that has been written so far? Would it be stupid idea to have a separate form, like `defn-r` that would require explicit `return`?

> Would it be stupid idea to have a separate form, like `defn-r` that would require explicit return?

That's pretty much what I suggested before with different names, though to be clear, the explicit-return form would still allow Python to implicitly return `None`. It would still compile without a `(return)` and Hy wouldn't insert a `return` statement.

> > Would it be stupid idea to have a separate form, like `defn-r` that would require explicit return?
> 
> That's pretty much what I suggested before with different names, though to be clear, the explicit-return form would still allow Python to implicitly return `None`. It would still compile without a `(return)` and Hy wouldn't insert a `return` statement.

What I tried to explain is that I think it would be good idea to let `fn` work like they do now, with implicit returns and add another form that requires explicit one. I know next release will break most of Hy programs ever written, but if we change how `fn` works, we're breaking pretty much each and every one. Although, I suppose it's possible to do search for `defn` and replace it with `define` + `lambda` combination.

But @gilch has an interesting point about how `def` and `lambda` work in Python in regards to needing return statement. Maybe I'm slowly changing my view on the subject after all.

Another option:

Keep the implicit tail return, but modify the compiler to suppress the `return` statement if `pass` is in the tail position. Add a `(return ...)` form to Hy that inserts a `return ...` statement. Thus both an implicit and explicit return could happen in the same form.

It's not that confusing; other languages (like Rust) have both explicit and implicit returns. I don't think an optional explicit `return` is any worse than the optional explicit `yield` we have now. No way we're getting rid of `yield`.

I like this latest proposal.

:+1: for @gilch's latest idea!

Three's a quorum/three's a veto. Unless someone has a better idea, we can push this through. 

But how exactly is this supposed to compile? I don't think Hy has ever had a `pass` form. It's obvious when there's only one exit, but what if there are multiple tails, like this?:

``` Hy
(defn foo [x]
  (if x
    pass
    "spam"))
```

I think this is a sensible output:

``` Python
def foo(x):
    if x:
        pass
    else:
        return 'spam'
```

But, given the way Hy already works it might be easier to do this:

``` Python
def foo(x):
    if x:
        pass
        _hy_anon_var_1 = None
    else:
        _hy_anon_var_1 = 'spam'
    return _hy_anon_var_1
```

But we still need to suppress the return when `pass` appears in all of the tails:

``` Hy
(defn foo [x]
  (if x
    pass
    pass))
```

``` Python
def foo(x):
    if x:
        pass
        _hy_anon_var_1 = None
    else:
        pass
        _hy_anon_var_1 = None
```

Now we have a wasted assignment. Hy already seems to do that though.

I'm thinking of the simple approach to this, which I don't think has been mentioned yet: add a `(return …)` form, which takes one optional argument and compiles to a `return` statement in the obvious way, but leave all the rules about implicit returns as they are (except that a form ending in `(return …)` need not have an implicit return added after it, of course). The implementation of this may lead to complications with our logic that turns off implicit returns when there's a `yield` in a function for old Pythons, but hopefully I can work around that.

Resolving patch:
diff --git a/hy/compiler.py b/hy/compiler.py
--- a/hy/compiler.py
+++ b/hy/compiler.py
@@ -322,21 +322,25 @@ def _raise_wrong_args_number(expression, error):
                                len(expression)))
 
 
+def _nargs(n):
+    return "%d argument%s" % (n, ("" if n == 1 else "s"))
+
+
 def checkargs(exact=None, min=None, max=None, even=None, multiple=None):
     def _dec(fn):
         def checker(self, expression):
             if exact is not None and (len(expression) - 1) != exact:
                 _raise_wrong_args_number(
-                    expression, "`%%s' needs %d arguments, got %%d" % exact)
+                    expression, "`%%s' needs %s, got %%d" % _nargs(exact))
             if min is not None and (len(expression) - 1) < min:
                 _raise_wrong_args_number(
                     expression,
-                    "`%%s' needs at least %d arguments, got %%d." % (min))
+                    "`%%s' needs at least %s, got %%d." % _nargs(min))
 
             if max is not None and (len(expression) - 1) > max:
                 _raise_wrong_args_number(
                     expression,
-                    "`%%s' needs at most %d arguments, got %%d" % (max))
+                    "`%%s' needs at most %s, got %%d" % _nargs(max))
 
             is_even = not((len(expression) - 1) % 2)
             if even is not None and is_even != even:
@@ -2312,6 +2316,16 @@ def compile_function_def(self, expression):
 
         return ret
 
+    @builds("return")
+    @checkargs(max=1)
+    def compile_return(self, expr):
+        ret = Result()
+        if len(expr) > 1:
+            ret += self.compile(expr[1])
+        return ret + ast.Return(value=ret.force_expr,
+                                lineno=expr.start_line,
+                                col_offset=expr.start_column)
+
     @builds("defclass")
     @checkargs(min=1)
     def compile_class_expression(self, expressions):


Test patch:
diff --git a/tests/native_tests/language.hy b/tests/native_tests/language.hy
--- a/tests/native_tests/language.hy
+++ b/tests/native_tests/language.hy
@@ -849,6 +849,66 @@
   (assert (= mooey.__name__ "mooey")))
 
 
+(defn test-return []
+
+  ; `return` in main line
+  (defn f [x]
+    (return (+ x "a"))
+    (+ x "b"))
+  (assert (= (f "q") "qa"))
+
+  ; Nullary `return`
+  (defn f [x]
+    (return)
+    5)
+  (assert (none? (f "q")))
+
+  ; `return` in `when`
+  (defn f [x]
+    (when (< x 3)
+      (return [x 1]))
+    [x 2])
+  (assert (= (f 2) [2 1]))
+  (assert (= (f 4) [4 2]))
+
+  ; `return` in a loop
+  (setv accum [])
+  (defn f [x]
+    (while True
+      (when (zero? x)
+        (return))
+      (.append accum x)
+      (-= x 1))
+    (.append accum "this should never be appended")
+    1)
+  (assert (none? (f 5)))
+  (assert (= accum [5 4 3 2 1]))
+
+  ; `return` of a `do`
+  (setv accum [])
+  (defn f []
+    (return (do
+      (.append accum 1)
+      3))
+    4)
+  (assert (= (f) 3))
+  (assert (= accum [1]))
+
+  ; `return` of an `if` that will need to be compiled to a statement
+  (setv accum [])
+  (defn f [x]
+    (return (if (= x 1)
+      (do
+        (.append accum 1)
+        "a")
+      (do
+        (.append accum 2)
+        "b")))
+    "c")
+  (assert (= (f 2) "b"))
+  (assert (= accum [2])))
+
+
 (defn test-mangles []
   "NATIVE: test mangles"
   (assert (= 2 ((fn [] (+ 1 1))))))