That is, they should have been here, to Earth, long before humans evolved. Even if, for whatever reason, they skipped past Earth, they should have been by some other star in the neighborhood. Since they have not, that implies perhaps that intelligent life is, for reasons we don't understand, a rare occurrence. Maybe that's true. SETI researchers repeatedly echo the sentiment first noted in the first scientific paper by Giuseppe Cocconi and Philip Morrison back in 1959.
Interstellar travel is so costly and impractical that the aliens never leave home.
SETI pioneer Frank Drake, for one, espouses this view. They just haven't gotten to this part of the galaxy yet. They came, built the Great Pyramids of Egypt and left. Or that there are thousands, maybe hundreds of thousands, of alien civilizations out there, and none of them developed interstellar travel. 100 billion stars.