By moving the ListAppender into the body of this particular test, it fixed all of the things.