We know we are not in a very large (observable-universe-sized) black hole, because with present technology we would notice a directional dependence on matter density and angle-brightness relations.

For such things, the standard cosmology's Friedmann-Lemaître-Walker-Robertson (FLRW) model is homogeneous and isotropic: everything's arranged around us spherically. A black hole model instead would tend to have everything arranged around us cylindrically, with objects like quasars and supernovae [or really, the arrangement of the spectral lines associated with them] along the axis redder and dimmer than similar things along the radius.

No universe-size black hole metric offers a decent explanation for the local physics of galaxy clusters. They're also poor descriptions: it is really hard to contrive a black hole metric which preserves the cosmological redshift for the bright parts of the observable universe, notably the direction-indepenent features of the Lyman-alpha forest. They also seriously struggle with the effectively flat spatial geometry of the observable universe clearly seen by the WMAP and Planck instruments (and in evidence as far back as BOOMERaNG/Maxima/TOCO/Saskatoon experiments in the 1990s) with support from morphology studies of high redshift galaxies.

In essence, using General Relativity to describe our universe, there is no large scale distortion associated with the Weyl curvature tensor to be found (or if you like, the gradient of the curvature density outside galaxy clusters is so negligibly small we can't even decide on current evidence which direction it grows in), and one of the chief characteristics of black hole metrics is that the Weyl tensor grows -- and ultimately diverges -- inside the horizon.

At large scales, our view of the sky is much better represented (and explained!) by the expanding FLRW metric of the standard cosmology, sprinkled with collapsing vacuoles to better capture the local physics of overdense regions (which usually contain galaxy clusters); this is called a "swiss-cheese model".

It would be hard not to notice being in a black hole whose radius isn't at least tens of orders of magnitude larger than the observable universe. We could think along the lines of embedding our cosmological Friedmann-Lemaître-Robertson-Walker metric in a black hole metric just as the FLRW metric surrounds a collapsing spacetime metric like Lemaître-Bondi-Tolman in "swiss-cheese model" (LTB is a type of black hole metric in which early gas and dust collapses spherically into a central black hole, while we'd prefer a type of metric where the gas and dust collapses in a more complicated way, evolving through star-filled galaxies and swarms of smaller black holes, before ultimately collapsing into a single gargantuan black hole). The FLRW metric would capture the observed cosmological effects induced in the vast and growing space around galaxy clusters, and "hide" the evidence of shrinking at much much larger distance scales. The problem is then how the real universe avoids giving us evidence that supports a recollapse of the expanding FLRW; this mostly means exploiting the low precision of the measurements we make when studying the shape of the universe.

Other embedding and inhomogeneous approaches are available, building on the complexity of my parenthetical in the previous paragraph, but all such approaches are likely suffer from the same problem: in order to be embedded in a collapsing spacetime, the inner expanding spacetime must be at all times very small in comparison. It is much easier to embed even an enormous collapsing spacetime into an expanding cosmology -- indeed, Einstein did just that in 1932 with de Sitter <https://en.wikipedia.org/wiki/Einstein%E2%80%93de_Sitter_uni...>, serving as a prototype of a homogenous "swiss-cheese model" for several decades before the discovery of the small temperature anisotropies of the cosmic microwave background.