You can visit the Caucasus year-round, but if you want to hike in the Caucasus mountains (especially Svaneti), aim for June through August.