The men were held in barracks to avoid distraction of their growth as a military unit to be reckoned with