Add guards for using whole directory mode in bad ways