If you instead ask, "could we fail to notice if a black hole was near enough to collide", the answer is probably not, at least in this epoch of the universe.

Most astrophysical black holes will tend to have accretion disks that are very bright.

Even if there are really isolated, almost non-accreting black holes in deep space, they will visibly distort (by Einstein lensing) the image of distant light sources behind, and there are many many many of those. We can already in principle use microlensing to find ones that are at the margins of the Milky Way, at distances of tens of kiloparsecs. They will in general be easier to spot at closer distances.

Eventually, thanks to the expansion of the universe, black holes in deep space might not have much to lense: background galaxies will stop being visible (many are already so red and dim that when Einstein-lensed by a black hole they will not be naked-eye visible), so you may be surprised by a black hole that is plainly visible if it is between you and the Milky-Way-Andromeda merged galaxy but has almost nothing to magnify when you are between it and the Milky-Way-Andromeda galaxy.
This is not something to worry about for the next many billion years.

Additionally, if you have reasonably sensitive instrumentation in your spaceship, you are even less likely to be surprised even if a black hole isn't at all naked-eye visible; black holes, even highly isolated ones in the distant future, will still interact with the relic fields like the Cosmic Microwave Background, and can be detected by those interactions.

> sucked into

Black holes don't reach out and grab things gravitationally, really. In general, if you treat them as Eulerian flow observers (as in fluid dynamics) they mostly deflect the trajectory of the "fluid" passing them (tangentially to the horizon), except for the fluid that flows directly onto the horizon or (tangentially) very close to it. In the case of a "fluid" of light, the deflection results in Einstein lensing, although light flowing close enough will be drawn into an orbit that (probably) ultimately leads inwards to the black hole. In the case of "fluids" that are massive -- gas, dust -- the speed at which the flow passes the black hole will always be less than that of light, but the flow is still deflected. Finally, in the case of larger objects like rocky planets or even stars, again their trajectories past the black hole will be deflected, and the result is similar to a hyperbolic orbit. Only the fluid which is aimed very well risks being swept up into an orbit, although some fluid can always be "aimed" directly at the black hole's notional surface (the event horizon), in which case it is on a one-way journey.

A "close call" in a space ship for a very large black hole might produce a large deflection and a significant time dilation, but otherwise is not especially dangerous. A close call for a small black hole might produce enough tidal stresses to compromise the structural integrity of the ship, but otherwise is also not especially dangerous. The more important dangers of black holes would be the intense radiation from their likely accretion disks, and again, that's readily detectable at long distances.

One could consider an analogy with a planet with an atmosphere: if, on a tangential course, you zip by the planet at sufficient distance, you get a pretty view. If you zip by at smaller distances, you might get what rocket scientists call a gravitational boost, but otherwise are probably fine. But if you zip by at smaller distances still, you might collide with the upper atmosphere, which might damage your spaceship (or incinerate it). You generally need an intersecting course to actually hit the rocky surface, although the atmosphere's effects may be relevant for non-intersecting courses. Black holes don't have a rocky surface, but the accretion structures around them can serve as a dangerous atmosphere.

In general, except in binary (and triple and so on) systems where the BH has a stellar partner, black holes tend to grow pretty slowly because not much falls directly on them, things that come close enough to have their trajectories bent into an orbit can stay in orbit for a long time, and not very much tends to come close enough to be drawn into an orbit in the first place. They don't act like the one shown in Disney's _The Black Hole_, for example (the human characters in that film would die of radiation poisoning, mostly, since their ships do not have many-metres-thick lead walls to block the X rays that would be streaming out of the bright accretion structure; turning off the Cygnus's magical station-keeping engine would mostly cause it to fall into a quasi-Keplerian orbit that could be stable for a very long time, so the film did get it kinda right that the Cygnus would need to be steered (with full engines!) directly towards the black hole).

Finally, this entertaining transcript is a bit more chatty-colloquial, and raises some additional points: https://www.nasa.gov/connect/chat/black_hole_chat.html (it might also raise some additional questions, though! For instance, "theoretically could stand at the event horizon point of a blackhole and not be sucked in" doesn't go into how to do this by e.g. constructing a scaffold around the BH that one could use as a platform to stand above it.)