Adding text to raster images involves incorporating text into the pixel data, leading to potential quality loss upon resizing. In contrast, adding text to a vector photo treats text as a separate object, allowing for scalability without compromising quality.