At the other end of the scale, there is such a thing as too small an exit pupil. The basic limit is imposed by the diffraction inherent in any telescope's aperture. Use a magnification of more than 50x per inch of aperture, even in perfect atmospheric seeing, and you're just magnifying diffraction fuzz. This means the minimum useful pupil size for any telescope is 0.5 mm. But even with a 1 mm pupil you're likely to notice some bothersome effects.
Floaters tend to increase with age.
Other fine irregularities may be visible too. Normally the neural network behind your retina does a remarkable job of image-processing all these things out of view. However, it tends to get thrown for a loop when the image is formed by narrow light cones from a tiny pupil. This is only logical. The eye never uses a 1 mm pupil in nature, so our visual processing system didn't evolve to fix whatever problems such a small pupil presents.