Yes, Matera is still worth visiting if it's raining. There are plenty of places to visit even if it's raining. Head indoors - inside the dramatic churches, into a restaurant that serves local specialties, inside one of the town's museums, or down into the underground cisterns.