Yes. With the fuse removed, this can easily be achieved by setting the meter to the lowest resistance setting and taking a reading across the (glass) fuse caps or termination prongs. A very low resistance reading, indicates continuity and the fuse is working, while a high reading is a sure sign the fuse is blown.