Rust 2020: Testing
By: Audun Halland
Tags: Teknologi, Programming, Testing, Rust

Rust is a different beast compared to the aforementioned languages. Being a system language focused on bare metal performance, abstractions do not “come for free”. At least not the ones I’m interested in here. First of all, all function calls are by default dispatched statically. This means that code calling another function directly is compiled into calling the exact implementation of that function, and not by a variable or table lookup. This is true also when calling regular methods of concrete types — struct methods are not overridable.

The two first techniques mentioned above can be applied perfectly fine in Rust, but there is a lot more for the developer to consider (and write!) in each case. The first thing to consider when introducing an abstraction is whether to use static or dynamic method dispatch. Static dispatch is related to monomorphization and generics. It means that a function A calling into an unknown function B is only lazily compiled — only when a call to some concrete B is known does the compiler emit code for A calling that exact B. This behaviour is optional, and we may choose to do dynamic dispatch instead, either by using a function pointer or by using a trait object, a way to call trait methods using a virtual lookup table passed along with the data pointer to the object instance.

So the way to express an abstraction in Rust is to use a trait. Trait mocking in Rust today is fairly developer friendly, but not as friendly as in dynamic-dispatch-languages. What I’ve been doing so far is to introduce a new trait everywhere I need test isolation, and use the crate mockall to autogenerate a mocked implementation that I instantiate in my test. Not very far from how java/mockito or jest works, except that, as mentioned, we always need to be very explicit about the abstraction taking place:

#[cfg_attr(test, mockall::automock)]

trait SomethingIWantToMock {

fn mockable();

} struct ConcreteSomethingUsedInProduction {} impl SomethingIWantToMock for ConcreteSomethingUsedInProduction {

fn mockable() { ... }

} #[test]

fn test_something() {

let mock_something = MockSomethingIWantToMock::new();

// test some code using that dependency

}

In Rust, making code dynamic dispatch is less verbose than making it static dispatch. This is in some ways counterintuitive, because in other areas of the language it appears to be a concious decision that less verbose code is simpler and therefore more efficient. Heap allocation is one example, it is always very apparent that a heap allocation will take place: Box<MyThing> instead of MyThing. A heap allocation is more expensive to type and execute. It seems concious that there is no short hand for this. First consider dynamic dispatch:

fn a(b: &dyn B) {

b.b();

}

where B is the trait a is abstract over. Now the static dispatch version of the above:

fn a<T: B>(b: T) {

b.b();

} // or alternatively

fn a<T>(b: T) where T: B {

b.b();

}

To make the function abstract with static dispatch (not losing runtime performance), we needed to introduce two new names. First B for the trait itself, and then the name T that represents its concrete type when a is monomorphized over it.

Generic code can be a lot of fun to write, but I wouldn’t like it if my codebase is full of it just for test isolation purposes. I want my business logic to be clear and readable when it does concrete things—generic code mostly belongs in utilities. And in principle, I don’t want to make all my code dynamic dispatch just because I want to test it. For all I know the code could be a really performance critical hot path in my application. And I did choose to write it in Rust in the first place.

Module mocks in Rust

How nice would it be to have module mocks in Rust? My very first thought was that it sounds technically infeasible. All tests in Rust run in a thread pool. In Rust it is forbidden to mutate global state, with good reason. There’s no sandboxing environment like in Jest. All mocked module functions would need to implicitly take a hidden argument—or the whole test executable would need to be “monomorphized” for each individual test case, effectively leading to N copies of the program where N is the number of tests…

But it turns out you can still do this, using the nightly compiler! Enter Mocktopus, a module-level mocking library. Using a macro to transform a module definition, mocktopus turns any public function in that module into a mockable by rewriting its definition into first checking (at call time) if a function pointer has been registered, which is stored in a thread-local lookup table. The macro only takes effect in a test build, not in debug or release. I think this technique could fit well with our project, but trying to be somewhat serious we will not depend on experimental nightly-only Rust features.

Unfortunately, my current solution is that I do not try to make my tests isolated where it is not strictly needed for correctness or where it would lead to awkward-to-read business logic code. The consequence of this is that many places in the code I essentially test the same thing over and over again. But this is also due to a completely different (slight) shortcoming in the Rust testing ecosystem: The basic assertion macros.

Assertion macros

My test assertions are exclusively variations of assert_eq! and assert_ne! . There are assertion utility libraries that one could pull in, but none of them seem to be very widely used, there is no clear adopted industry standard on top of std::assert . Let’s go through some very simple code to highlight the slight inconvenience:

struct Answer {

foo: Vec<String>,

bar: Vec<String>

} fn compute_answer(some_input: &str) -> Answer {

Answer {

foo: some_input.split("."),

bar: external_module::compute_bar(some_input)

}

}

Testing this function in isolation is the same as checking if the input string is split correctly. The test of external_module belongs in a different file, so we should ignore the value of bar in our test. But:

#[test]

fn test() {

assert_eq!(

compute_answer("input.string"),

Answer {

foo: vec!["input".to_string(), "string".to_string()],

bar: vec!["?"]

}

);

}

using assert_eq! I have to compare the full struct. Of course I could compare the member(s) explicitly:

#[test]

fn test() {

let aswer = compute_answer("input.string");

assert_eq!(

answer.foo,

vec!["input".to_string(), "string".to_string()],

);

}

but it’s not as elegant as having a single matching expression for the whole type that I’m testing. But what if there was a macro for matching only what I care about? Let’s try the assert_matches! macro from the matches crate, a macro that makes it possible to assert that something matches a pattern similar to a match expression:

#[test]

fn test() {

assert_matches!(

compute_answer("input.string"),

Answer {

foo: ["input", "string"], // Does not compile

bar: _, // Actually works!

}

);

}

AFAIK it’s not possible to match a struct deeply like this, because directly matching a Vec to a slice pattern does not work. Only things that are already slices may be pattern matched, and the members of Answer are not slices but Vecs, and not coerced.