Make sure ./plugins exists before looking through it