How to abstract away library interfaces working directly through syntax in Rust with procedural macros
Tags: rust, cleancode, tutorial
Published: 2023-09-21 11:41:32

In this post I describe how I wrote two macros to separate the [bevy_ecs](https://docs.rs/bevy_ecs/latest/bevy_ecs/) interface from my app core even though bevy_ecs' system interface works directly via the function syntax. 
This isn't a complete tutorial for Rust's syn and quote libraries, the Entity Component System pattern or Rust macros. I won't explain every struct that I use. Read the library documentations for that.

You can find my source code [here](https://github.com/pgurtner/animal_game_ecs_macros).

## general info about procedural macros
In both cases I'm using procedural macros to remain flexible. With bevy_ecs a derive Component macro would probably suffice, but some ECS libraries presumably use another way which would cause problems if I tried to switch to them.

Procedural macros get two token streams (one for the macro arguments like `Debug` in `#[derive(Debug)]`, one for the block that has the macro) and return one (the code that replaces the code block which is affected by the macro). They look like this:
```rust
#[proc_macro_attribute]
fn macro_name (args: TokenStream, input: TokenStream) -> TokenStream {
    //...
}
```

I haven't used macro arguments yet, so I won't describe them.

To parse the incoming token streams into something usable like a token tree, Rust offers the [syn](https://docs.rs/syn/latest/syn/) package. For the opposite direction Rust offers the [quote](https://docs.rs/quote/latest/quote/) package.

To get from the token stream to a token tree, syn has multiple parsing macros. I used `parse_macro_input!`. The main difference to the others is that `parse_macro_input!` enforces what type of syntax block you get. E.g. `parse_quote!` just parses any string.
A call would look like this: 
```rust
let parsed_input = parse_macro_input!(input as ItemStruct);
```
In this case my custom macro is expecting a struct. I parse the `input` parameter to `ItemStruct` and `parse_macro_input!` checks if `input` actually is a struct and returns an `ItemStruct` instance. `ItemStruct` is a normal Rust struct provided by syn that can be nicely worked with!

After you've edited your token tree or created a new one you have to create a new token stream that will be the output of the custom macro. There are multiple ways for different scenarios.
The most simple one is using the `quote!` macro. 
```rust
let parsed_input = something_that_implements_ToTokens;
let input_struct_identifier = again_something_that_implements_ToTokens;

let new_tokenstream = quote! {
    #parsed_input
    
    impl #input_struct_identifier {
        fn im_just_here_to_show_what_quote_can_do (a: i32) -> bool {
            a < 100_00
        }
    }
};
```

In `quote!` you can directly write Rust syntax and use variables that implement the `ToTokens` trait (`parsed_input` and `input_struct_identifier` in this example). `quote!` substitutes these variables with their token streams.
The example above could produce a token stream resembling something like this:
```rust
struct SomeStruct {
    //...
}

impl SomeStruct {
    fn im_just_here_to_show_what_quote_can_do (a: i32) -> bool {
        a < 100_00
    }
}
```

However, if you created a struct that already implements the `ToToken` trait you can directly parse it into a token stream.
E.g. like that:
```rust
item_struct.into_token_stream()
```

## component macro
The component macro is rather simple. bevy_ecs uses the same style that I want, so the macro simply adds a `#[derive(bevy_ecs::component::Component)]`.

```rust
let expanded = quote! {
    #[derive(bevy_ecs::component::Component)]
    #input
};
```

There is probably a way to add the derive macro in the `ItemStruct` instance, but I didn't try that.

## system macro
The system macro is far more complex and interesting.
In general, I want my system functions like this:
```rust
#[system]
fn some_system (a: &ComponentA, b: &mut ComponentB, entity: Entity) {
    //a,b and entity are used here
}
```

bevy_ecs on the other hand, uses this style:
```rust
fn some_system (query: Query<&ComponentA, &mut ComponentB, Entity>) {
    for (a, mut b, entity) in &mut query {
        //a, b and entity are used here
    }
}
```

This has quite some syntax overhead that I don't like. Thus, I combined keeping dependencies away from my program with getting an interface more of my liking.

This comparison already shows how the token stream input looks like and how the output is supposed to be.

The macro can be seperated in creating a new function signature and creating the for loop. Then you create a new function that copies all the other properties from the old function and insert the new signature and new function body.
It could look like this:
```rust
pub fn create_fn_item (input: ItemFn) -> Result<ItemFn, TokenStream> {
    let new_sig = create_new_sig(&input.sig)?;
    let for_loop_stmts = create_new_loop_stmts(&input)?;

    let new_fn = ItemFn {
        attrs: input.attrs, // attributes like #[repr(transparent)]
        vis: input.vis, // visibility
        sig: new_sig, // signature
        block: Box::new(syn::Block{ // function body
            brace_token: Default::default(),
            stmts: for_loop_stmts
        }),
    };

    return Ok(new_fn);
}
```

I won't explain the steps for the signature and the for loop in detail because it would mainly be copying source code and writing some redundant comment. My biggest advise is to parse the structures you want and print them to the console. Then look how they are build up and combined with the syn documentation build your wanted syntax structures. You can use my [source code](https://github.com/pgurtner/animal_game_ecs_macros/blob/main/src/bevy_system.rs) as an example.

## debugging procedural macros
Currently procedural macros can only be executed in their default way with `#macro_name`, even though they're just special functions. This means quite some problems for debugging because macros are executed at compile time. There are some debugging options but not a usual IDE debugger.
However, there is a workaround for that. The library [proc_macro2](https://docs.rs/proc-macro2/latest/proc_macro2/). This is a wrapper around the procedural macro API, and it can be used outside a procedural macro. That means you can outsource your macro logic into plain functions and call these from your macro. The actual macro only does the initial syntax parsing, the conversion between proc_macro and proc_macro2 and calls the outsourced function. Because the latter is a plain and normal Rust function, you can unit test it or do whatever else you want.
This is an example how it might look:
```rust
#[proc_macro_attribute]
pub fn to_bevy_component(
_args: TokenStream,
input: TokenStream,
) -> TokenStream {
    let input = parse_macro_input!(input as ItemStruct); 

    let result = bevy_component::to_bevy_component(input);

    proc_macro::TokenStream::from(result)
}
```

```rust
pub fn to_bevy_component (input: ItemStruct) -> proc_macro2::TokenStream {
    let expanded = quote! {
        #[derive(bevy_ecs::component::Component)]
        #input
    };

    return expanded;
}

#[cfg(test)]
mod tests {
    //...
}
```

## throwing errors in procedural macros
In case I mess up and use a macro on some invalid syntax thing I wanted error management via `Result` like usual. This doesn't mean syntactical problems but e.g. some syntax thing that isn't supposed to appear in the syntax things affected by the macro and that I therefore don't want to have to handle.
You can do it quite easily. You just have to create a `syn::Error`, call `.into_compile_error()` on it, parse it into a token stream and return it from your macro. For Rust to be able to determine the right spot that caused the error, you have to get the correct span. A span is some part of your parsed code that starts and ends somewhere. Usually you can get it from the syn structs.

I used this technique to exclude self parameters, because systems will never be associated to a struct (or at least don't offer any additional value than normal functions).
```rust
 match fn_arg {
        FnArg::Receiver(rec) => Err(
            syn::Error::new(rec.self_token.span, "systems can't have a self parameter")
                .into_compile_error()
        ),
    //...
```