Not necessarily. While yes, there are dark fantasy stories that will dive headfirst into the horror aspects of things, there are others that just use it as a setting for mystery, like many of Neil Gaiman's works which feature fantastical worlds with small tinges of darkness lurking around the corner.