I don't have anything against fantasy generally. I enjoy it and I publish it, but it is useful to hold that they are two different things. For the purpose of the rest of this essay, I will use the word 'mainstream' to mean everything not SF in the way described above. But we can say that generally there exists a collection of fictions that does not deliberately evoke the sorts of variations from our daily reality described above. Which then brings us to the question of whether a story can be both 'literary' and 'science fiction' at the same time.
Escapism is fiction that encourages the supression of the critical faculties.
Which is not necessarily a value judgement in itself. Any sort of fiction is going to have many examples which are just plain dull, stupid, and badly written. We all know what paves the road to hell. And of all literary values, entertainment value is certainly the most subjective, the least tractable to analysis. But it seems fair to say that any fiction should have some, to somebody.