What Constitutes a Secondary Dominant?

A secondary dominant chord has chromatic notes in it - it's non-diatonic. As in key C, E7 can be called a secondary dominant chord. BUT to be a dominant, or secondary dominant, does it HAVE to lead to the chord it's actually dominant to? Bearing in mind that, I guess, G is always called 'dominant', even though it may not lead straight to the tonic, which is what it often does. Often, but not always.
That E7 will often, if not generally, be followed by A or Am, thus making it the dominant of that A chord. Thus, secondary dominant would appear to be a logical name for it in those cicumstances.
But, if it was followed by, say, an F chord, as sometimes happens, would it still be labelled secondary dominant, as it actually doesn't fulfil that role in the music.
If so, why? And if not, what name would it be given?

No, it doesn't have to lead to the chord it's actually dominant to. (Keep in mind that "dominant" doesn't really mean "leads to tonic," it just means "built a fifth above tonic.")
Occasionally these secondary dominants will resolve deceptively, like in your E7–F example. Really, this is V7–VI within a small pocket of A minor that takes place in a more global context of C major.
Here's an example I used in a course recently; it's the overture to Wagner's Tannhäuser:

This is in E major (and it modulates, or at least tonicizes, B). In m. 12, we have a D♯7 chord (V7/iii). It doesn't resolve to G♯, but rather it resolves deceptively, creating a V7–VI motion in the key of G♯ minor. Furthermore, the C♯-minor chord at the end of m. 11 is a iv6 in G♯. As such, I understand this as what we call an extended tonicization in the key of G♯. Instead of just a V–i tonicization, we include multiple chords here: in a local section of iii (G♯), we have the progression iv6–V7–VI.
In other instances, you can have what some call a "back-relating dominant," where the dominant actually appears after its respective tonic. In C major, you might have C–Am–E7 before moving on to something else. But still this can be understood as vi moving to V7/vi.
But in still other cases, an E-major chord in the context of C might just be a more advanced chromatic coloration, and it might be best understood as a III♯ chord. It's really all about context.