Yes, you should always wash cilantro and other produce. Hold the stems and rinse under running water or dunk in a bowl of cool water. Shake off the excess water and let it dry on a towel before using.