Using relevant directories is crucial in directory link building because it ensures that your website is being listed in directories that are related to your industry or niche, making the links more valuable for SEO purposes.