Simplification often results in telling the truth, but not the whole truth. A perfect lens would produce a perfect image. However lenses are not perfect. Their defects are called abberations. There are 3 aberrations which directly lower the sharpness of an image, Spherical Aberration , Astigmatism and Coma .
Some lenses use aspheric elements to better correct spherical aberration at wide apertures.
Spherical aberration affects the whole image, astigmatism and coma mostly affect the edges and corners. A fifth aberration, Field Curvature , results in the image from the lens being formed as a curved suface, not flat like film, so when the center of the image is in focus, the edges are not, and vice-versa. Real light is polychromatic, i.e. made up of many colors which results in a sixth aberration, Chromatic Aberration . In theory , with enough effort and expense, all these aberations could be eliminated, or at least reduced to a point at which their effect would be very small. The smaller the aperture, the more diffraction and the lower the sharpness of the image.