Apparently cucumber cli doesn't work like that...