Rust Can't Handle Some Lifetimes of Generic Closure Arguments

Posted

I’ve been using a custom HTML generation library for Rust. All the other libraries that I found use some sort of DSL which is both hard to learn and can be limiting. I decided to make a simple framework to spit out HTML. This means that all of your templates are just regular code.

Having your templates be regular code means that you can create layouts, partials or whatever you want to call them using regular functions. For example, I have a simple navigation generator:

async fn nav(parent: &mut Element) -> Result<(), Error> {
	let mut nav = parent.child("nav").await?;
	let mut list = nav.child("ul").await?;
	list_link(&mut list, "Account", crate::user_get_url()).await?;
	list_link(&mut list, "Subscriptions", crate::subscriptions_list_url()).await?;
	list_link(&mut list, "Help", crate::docs::faq_url()).await?;
	list.close().await?;
	nav.close().await
}

This helper also uses the list_link “partial” with a simple function call. This is a pretty simple template because it doesn’t accept any contents from the caller. For that a function parameter can used (or multiple, or a trait with many methods, or whatever else you want to use).

async fn form<
	Fut: std::future::Future<Output=Result<(), Error>>,
>(
	parent: &mut Element<'_>,
	submit_msg: &str,
	body: impl FnOnce(&mut Element) -> Fut,
) -> Result<(), Error> {
	let mut f = parent.child("form").await?;
	body(&mut f).await?;
	f.child("button").await?.text(submit_msg).await?;
	f.close().await
}

This type signature is a bit complex, especially if you are new to Rust. But basically what it is saying is that body is a function that can be called once (FnOnce). It takes a mutable reference to an Element and returns a future that will eventually evaluate to Result<(), Error> (either nothing, or an error).

One thing to note is that there is no allocation required. The FnOnce and Fut types are both decided by the caller and can be stored on the stack. This makes this implementation very efficient and very optimizer transparent. In theory this is a zero-cost abstraction.

However, it turns out that the lifetimes in this example are too restrictive. Let’s try making a form and seeing the problem.

form(
	parent,
	"Add TODO",
	|f| async move {
		f.child("input").await?.close().await
	}).await

Try it Yourself

This doesn’t compile.

error: lifetime may not live long enough
          |f| async move {
 __________---_^
|          | |
|          | return type of closure `impl Future<Output = [async output]>` contains a lifetime `'r`
|          has type `&'a mut Element<'_>`
|             f.child("input").await?.close().await
|         }).await
|_________^ returning this value requires that `'a` must outlive `'r`

The key info from the error is the following:

It is complaining that f is moved into the returned async block. But form() only tells us that f is alive for the function call. Referencing f in the returned value is forbidden because f may not longer exist once the function returns, even though the returned value still will. For example imagine the body of form() instead contained:

let mut f = parent.child("form").await?;
let future = body(&mut f);
f.close().await?; // f's lifetime ends here.
future.await // The returned future is used here.

This demonstrates why Fut isn’t allowed to reference f. The type signature of form() doesn’t guarantee that f lives long enough.

This means that you can’t really use f for anything (because most of f’s methods are async, so can only really be called inside a future). Updating the signature to allow body to hold a reference to f in the returned future requires HRTB to specify that the returned value of the body function is allowed to reference the argument we pass it.

async fn form<
	Fut: std::future::Future<Output=Result<(), Error>>,
>(
	parent: &mut Element<'_>,
	submit: &str,
	body: impl for<'arg> FnOnce(&'arg mut Element) -> 'arg + Fut,
) -> Result<(), Error>

But this isn’t allowed:

error[E0404]: expected trait, found type parameter `Fut`

body: impl for<'arg> FnOnce(&'arg mut Element) -> 'arg + Fut,
                                                         ^^^ not a trait

We need to describe Fut’s lifetime when it is declared. But this prevents us from using HRTB. If we try it looks something like this:

async fn form<
	'arg,
	'caller: 'arg,
	Fut: 'arg + std::future::Future<Output=Result<(), Error>>,
>(
	parent: &'caller mut Element<'_>,
	submit: &'caller str,
	body: impl FnOnce(&'arg mut Element) -> Fut,
) -> Result<(), Error>

But this isn’t correct, and form() doesn’t compile.

error[E0597]: `f` does not live long enough

    'arg,
    ---- lifetime `'arg` defined here

    body(&mut f).await?;
    -----^^^^^^-
    |    |
    |    borrowed value does not live long enough
    argument requires that `f` is borrowed for `'arg`

}
- `f` dropped here while still borrowed

The problem here is that having 'arg as a lifetime parameter means that the caller gets to decide what the lifetime is. That is why it is called a parameter, it is an input to the function. However, for this use case we want to pick the lifetime, that is why we need HRTB.

As far as I am aware there is no way to describe this function in Rust. It would be easy to do if impl was allowed in the return type of FnOnce like follows:

async fn form(
	parent: &mut Element<'_>,
	submit: &str,
	body: impl for<'arg>
		FnOnce(&'arg mut Element)
		-> (impl 'arg + std::future::Future<Output=Result<(), Error>>),
) -> Result<(), Error>

This way the 'arg lifetime is in-scope for the definition of the return value. This would effectively add a new type parameter to the function just like the other impl is doing.

Workaround

The work-around is to use runtime polymorphism which effectively requires an extra allocation. The result looks like:

async fn form(
	parent: &mut Element<'_>,
	submit: &str,
	body: impl for<'arg>
		FnOnce(&'arg mut Element)
		-> std::pin::Pin<Box<
			dyn 'arg + std::future::Future<Output=Result<(), Error>>>>,
) -> Result<(), Error>

And can be used by wrapping the returned future in a Box::pin call.

form(
	parent,
	"Delete",
	|f| Box::pin(async move {
		f.child("p").await?.text("Delete this entry?").await
	})).await