Yes, you can. Great idea, in fact. This furnace troubleshooting step will help you rule out whether the igniter is the cause of the gas furnace not igniting or starting.
Our guide “How to Tell if Your Furnace Ignitor Is Bad & Where to Buy One” has step-by-step instructions that are easy to follow, if you need them. Perhaps if you already own a multimeter, you know how to test an igniter.