Rust Can't Handle Some Lifetimes of Generic Closure Arguments
Tags: rust
Published: 2022-05-08 11:39:05

---
title: Rust Can't Handle Some Lifetimes of Generic Closure Arguments
published: true
date: 2022-04-16 13:00:00 UTC
tags: Rust
canonical_url: https://kevincox.ca/2022/04/16/rust-generic-closure-lifetimes/
---

<main><p>I’ve been using a custom HTML generation library for Rust. All the other libraries that I found use some sort of <abbr title="Domain Specific Language">DSL</abbr> which is both hard to learn and can be limiting. I decided to make a simple framework to spit out HTML. This means that all of your templates are just regular code.</p>
<p>Having your templates be regular code means that you can create layouts, partials or whatever you want to call them using regular functions. For example, I have a simple navigation generator:</p>
<div><pre><code><span>async</span> <span>fn</span> <span>nav</span><span>(</span><span>parent</span><span>:</span> <span>&amp;</span><span>mut</span> <span>Element</span><span>)</span> <span>-&gt;</span> <span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span> <span>{</span>
	<span>let</span> <span>mut</span> <span>nav</span> <span>=</span> <span>parent</span><span>.child</span><span>(</span><span>"nav"</span><span>)</span><span>.await</span><span>?</span><span>;</span>
	<span>let</span> <span>mut</span> <span>list</span> <span>=</span> <span>nav</span><span>.child</span><span>(</span><span>"ul"</span><span>)</span><span>.await</span><span>?</span><span>;</span>
	<span>list_link</span><span>(</span><span>&amp;</span><span>mut</span> <span>list</span><span>,</span> <span>"Account"</span><span>,</span> <span>crate</span><span>::</span><span>user_get_url</span><span>())</span><span>.await</span><span>?</span><span>;</span>
	<span>list_link</span><span>(</span><span>&amp;</span><span>mut</span> <span>list</span><span>,</span> <span>"Subscriptions"</span><span>,</span> <span>crate</span><span>::</span><span>subscriptions_list_url</span><span>())</span><span>.await</span><span>?</span><span>;</span>
	<span>list_link</span><span>(</span><span>&amp;</span><span>mut</span> <span>list</span><span>,</span> <span>"Help"</span><span>,</span> <span>crate</span><span>::</span><span>docs</span><span>::</span><span>faq_url</span><span>())</span><span>.await</span><span>?</span><span>;</span>
	<span>list</span><span>.close</span><span>()</span><span>.await</span><span>?</span><span>;</span>
	<span>nav</span><span>.close</span><span>()</span><span>.await</span>
<span>}</span>
</code></pre></div>
<p>This helper also uses the <code>list_link</code> “partial” with a simple function call. This is a pretty simple template because it doesn’t accept any contents from the caller. For that a function parameter can used (or multiple, or a trait with many methods, or whatever else you want to use).</p>
<div><pre><code><span>async</span> <span>fn</span> <span>form</span><span>&lt;</span>
	<span>Fut</span><span>:</span> <span>std</span><span>::</span><span>future</span><span>::</span><span>Future</span><span>&lt;</span><span>Output</span><span>=</span><span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;&gt;</span><span>,</span>
<span>&gt;</span><span>(</span>
	<span>parent</span><span>:</span> <span>&amp;</span><span>mut</span> <span>Element</span><span>&lt;</span><span>'_</span><span>&gt;</span><span>,</span>
	<span>submit_msg</span><span>:</span> <span>&amp;</span><span>str</span><span>,</span>
	<span>body</span><span>:</span> <span>impl</span> <span>FnOnce</span><span>(</span><span>&amp;</span><span>mut</span> <span>Element</span><span>)</span> <span>-&gt;</span> <span>Fut</span><span>,</span>
<span>)</span> <span>-&gt;</span> <span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span> <span>{</span>
	<span>let</span> <span>mut</span> <span>f</span> <span>=</span> <span>parent</span><span>.child</span><span>(</span><span>"form"</span><span>)</span><span>.await</span><span>?</span><span>;</span>
	<span>body</span><span>(</span><span>&amp;</span><span>mut</span> <span>f</span><span>)</span><span>.await</span><span>?</span><span>;</span>
	<span>f</span><span>.child</span><span>(</span><span>"button"</span><span>)</span><span>.await</span><span>?</span><span>.text</span><span>(</span><span>submit_msg</span><span>)</span><span>.await</span><span>?</span><span>;</span>
	<span>f</span><span>.close</span><span>()</span><span>.await</span>
<span>}</span>
</code></pre></div>
<p>This type signature is a bit complex, especially if you are new to Rust. But basically what it is saying is that <code>body</code> is a function that can be called once (<code>FnOnce</code>). It takes a mutable reference to an <code>Element</code> and returns a future that will eventually evaluate to <code><span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span></code> (either nothing, or an error).</p>
<p>One thing to note is that there is no allocation required. The <code>FnOnce</code> and <code>Fut</code> types are both decided by the caller and can be stored on the stack. This makes this implementation very efficient and very optimizer transparent. In theory this is a zero-cost abstraction.</p>
<p>However, it turns out that the lifetimes in this example are too restrictive. Let’s try making a form and seeing the problem.</p>
<div><pre><code><span>form</span><span>(</span>
	<span>parent</span><span>,</span>
	<span>"Add TODO"</span><span>,</span>
	<span>|</span><span>f</span><span>|</span> <span>async</span> <span>move</span> <span>{</span>
		<span>f</span><span>.child</span><span>(</span><span>"input"</span><span>)</span><span>.await</span><span>?</span><span>.close</span><span>()</span><span>.await</span>
	<span>})</span><span>.await</span>
</code></pre></div>
<p><a href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2021&amp;gist=56003ecf91dd7216b30bb6422ba9ef08">Try it Yourself</a></p>
<p>This doesn’t compile.</p>
<div><pre><code>error: lifetime may not live long enough
          |f| async move {
 __________ ---_^
| | |
| | return type of closure `impl Future&lt;Output = [async output]&gt;` contains a lifetime `'r`
| has type `&amp;'a mut Element&lt;'_&gt;`
| f.child("input").await?.close().await
| }).await
| _________ ^ returning this value requires that `'a` must outlive `'r`
</code></pre></div>
<p>The key info from the error is the following:</p>
<ul>
<li>The argument <code>f</code> has type <code><span>&amp;</span><span>'a</span> <span>mut</span> <span>Element</span><span>&lt;</span><span>'_</span><span>&gt;</span></code>.</li>
<li>The return type of closure <code><span>impl</span> <span>Future</span><span>&lt;</span><span>Output</span> <span>=</span> <span>[</span><span>async</span> <span>output</span><span>]</span><span>&gt;</span></code> contains a lifetime <code><span>'r</span></code>.</li>
<li>Returning the async block value requires that <code><span>'a</span></code> must outlive <code><span>'r</span></code>
</li>
</ul>
<p>It is complaining that <code>f</code> is moved into the returned <code><span>async</span></code> block. But <code><span>form</span><span>()</span></code> only tells us that <code>f</code> is alive for the function call. Referencing <code>f</code> in the returned value is forbidden because <code>f</code> may not longer exist once the function returns, even though the returned value still will. For example imagine the body of <code><span>form</span><span>()</span></code> instead contained:</p>
<div><pre><code><span>let</span> <span>mut</span> <span>f</span> <span>=</span> <span>parent</span><span>.child</span><span>(</span><span>"form"</span><span>)</span><span>.await</span><span>?</span><span>;</span>
<span>let</span> <span>future</span> <span>=</span> <span>body</span><span>(</span><span>&amp;</span><span>mut</span> <span>f</span><span>);</span>
<span>f</span><span>.close</span><span>()</span><span>.await</span><span>?</span><span>;</span> <span>// f's lifetime ends here.</span>
<span>future</span><span>.await</span> <span>// The returned future is used here.</span>
</code></pre></div>
<p>This demonstrates why <code>Fut</code> isn’t allowed to reference <code>f</code>. The type signature of <code><span>form</span><span>()</span></code> doesn’t guarantee that <code>f</code> lives long enough.</p>
<p>This means that you can’t really use <code>f</code> for anything (because most of <code>f</code>’s methods are async, so can only really be called inside a future). Updating the signature to allow <code>body</code> to hold a reference to <code>f</code> in the returned future requires <a href="https://doc.rust-lang.org/nomicon/hrtb.html">HRTB</a> to specify that the returned value of the <code>body</code> function is allowed to reference the argument we pass it.</p>
<div><pre><code><span>async</span> <span>fn</span> <span>form</span><span>&lt;</span>
	<span>Fut</span><span>:</span> <span>std</span><span>::</span><span>future</span><span>::</span><span>Future</span><span>&lt;</span><span>Output</span><span>=</span><span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;&gt;</span><span>,</span>
<span>&gt;</span><span>(</span>
	<span>parent</span><span>:</span> <span>&amp;</span><span>mut</span> <span>Element</span><span>&lt;</span><span>'_</span><span>&gt;</span><span>,</span>
	<span>submit</span><span>:</span> <span>&amp;</span><span>str</span><span>,</span>
	<span>body</span><span>:</span> <span>impl</span> <span>for</span><span>&lt;</span><span>'arg</span><span>&gt;</span> <span>FnOnce</span><span>(</span><span>&amp;</span><span>'arg</span> <span>mut</span> <span>Element</span><span>)</span> <span>-&gt;</span> <span>'arg</span> <span>+</span> <span>Fut</span><span>,</span>
<span>)</span> <span>-&gt;</span> <span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span>
</code></pre></div>
<p>But this isn’t allowed:</p>
<div><pre><code>error[E0404]: expected trait, found type parameter `Fut`

body: impl for&lt;'arg&gt; FnOnce(&amp;'arg mut Element) -&gt; 'arg + Fut,
                                                         ^^^ not a trait
</code></pre></div>
<p>We need to describe <code>Fut</code>’s lifetime when it is declared. But this prevents us from using HRTB. If we try it looks something like this:</p>
<div><pre><code><span>async</span> <span>fn</span> <span>form</span><span>&lt;</span>
	<span>'arg</span><span>,</span>
	<span>'caller</span><span>:</span> <span>'arg</span><span>,</span>
	<span>Fut</span><span>:</span> <span>'arg</span> <span>+</span> <span>std</span><span>::</span><span>future</span><span>::</span><span>Future</span><span>&lt;</span><span>Output</span><span>=</span><span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;&gt;</span><span>,</span>
<span>&gt;</span><span>(</span>
	<span>parent</span><span>:</span> <span>&amp;</span><span>'caller</span> <span>mut</span> <span>Element</span><span>&lt;</span><span>'_</span><span>&gt;</span><span>,</span>
	<span>submit</span><span>:</span> <span>&amp;</span><span>'caller</span> <span>str</span><span>,</span>
	<span>body</span><span>:</span> <span>impl</span> <span>FnOnce</span><span>(</span><span>&amp;</span><span>'arg</span> <span>mut</span> <span>Element</span><span>)</span> <span>-&gt;</span> <span>Fut</span><span>,</span>
<span>)</span> <span>-&gt;</span> <span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span>
</code></pre></div>
<p>But this isn’t correct, and <code><span>form</span><span>()</span></code> doesn’t compile.</p>
<div><pre><code>error[E0597]: `f` does not live long enough

    'arg,
    ---- lifetime `'arg` defined here

    body(&amp;mut f).await?;
    -----^^^^^^-
    | |
    | borrowed value does not live long enough
    argument requires that `f` is borrowed for `'arg`

}
- `f` dropped here while still borrowed
</code></pre></div>
<p>The problem here is that having <code><span>'arg</span></code> as a lifetime parameter means that the caller gets to decide what the lifetime is. That is why it is called a <strong>parameter</strong>, it is an input to the function. However, for this use case we want to pick the lifetime, that is why we need HRTB.</p>
<p>As far as I am aware there is no way to describe this function in Rust. It would be easy to do if <code><span>impl</span></code> was allowed in the return type of <code><span>FnOnce</span></code> like follows:</p>
<div><pre><code><span>async</span> <span>fn</span> <span>form</span><span>(</span>
	<span>parent</span><span>:</span> <span>&amp;</span><span>mut</span> <span>Element</span><span>&lt;</span><span>'_</span><span>&gt;</span><span>,</span>
	<span>submit</span><span>:</span> <span>&amp;</span><span>str</span><span>,</span>
	<span>body</span><span>:</span> <span>impl</span> <span>for</span><span>&lt;</span><span>'arg</span><span>&gt;</span>
		<span>FnOnce</span><span>(</span><span>&amp;</span><span>'arg</span> <span>mut</span> <span>Element</span><span>)</span>
		<span>-&gt;</span> <span>(</span><span>impl</span> <span>'arg</span> <span>+</span> <span>std</span><span>::</span><span>future</span><span>::</span><span>Future</span><span>&lt;</span><span>Output</span><span>=</span><span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;&gt;</span><span>),</span>
<span>)</span> <span>-&gt;</span> <span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span>
</code></pre></div>
<p>This way the <code><span>'arg</span></code> lifetime is in-scope for the definition of the return value. This would effectively add a new type parameter to the function just like the other <code><span>impl</span></code> is doing.</p>
<h2 id="workaround"><a href="https://kevincox.ca/2022/04/16/rust-generic-closure-lifetimes/#workaround">Workaround</a></h2>
<p>The work-around is to use runtime polymorphism which effectively requires an extra allocation. The result looks like:</p>
<div><pre><code><span>async</span> <span>fn</span> <span>form</span><span>(</span>
	<span>parent</span><span>:</span> <span>&amp;</span><span>mut</span> <span>Element</span><span>&lt;</span><span>'_</span><span>&gt;</span><span>,</span>
	<span>submit</span><span>:</span> <span>&amp;</span><span>str</span><span>,</span>
	<span>body</span><span>:</span> <span>impl</span> <span>for</span><span>&lt;</span><span>'arg</span><span>&gt;</span>
		<span>FnOnce</span><span>(</span><span>&amp;</span><span>'arg</span> <span>mut</span> <span>Element</span><span>)</span>
		<span>-&gt;</span> <span>std</span><span>::</span><span>pin</span><span>::</span><span>Pin</span><span>&lt;</span><span>Box</span><span>&lt;</span>
			<span>dyn</span> <span>'arg</span> <span>+</span> <span>std</span><span>::</span><span>future</span><span>::</span><span>Future</span><span>&lt;</span><span>Output</span><span>=</span><span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;&gt;&gt;&gt;</span><span>,</span>
<span>)</span> <span>-&gt;</span> <span>Result</span><span>&lt;</span><span>(),</span> <span>Error</span><span>&gt;</span>
</code></pre></div>
<p>And can be used by wrapping the returned future in a <code>Box::pin</code> call.</p>
<div><pre><code><span>form</span><span>(</span>
	<span>parent</span><span>,</span>
	<span>"Delete"</span><span>,</span>
	<span>|</span><span>f</span><span>|</span> <span>Box</span><span>::</span><span>pin</span><span>(</span><span>async</span> <span>move</span> <span>{</span>
		<span>f</span><span>.child</span><span>(</span><span>"p"</span><span>)</span><span>.await</span><span>?</span><span>.text</span><span>(</span><span>"Delete this entry?"</span><span>)</span><span>.await</span>
	<span>}))</span><span>.await</span>
</code></pre></div></main>