Don't use swift if no swift

Otherwise it fail a lot of tests