Add unit test for starting app

Unit test fix for #3675.