The application program doesn't have to worry about about all the different types of video display hardware. There is one common standard way of talking to all of them. The VGA display adapter has its own memory for storing the screen contents. This is completely separate from the main memory of the computer. For example, you might have 4 or 8 megabytes of main memory but the video display might have only 1 megabyte, or 512k, or even less.
Higher resolution needs more video memory.
More colors needs more video memory. There is a tradeoff. If you have 512k or less, you should consider adding more, if possible, or upgrading to a better model. If you have 1 megabyte of video memory and can get 32k or 64k colors at 800 x 600, don't feel compelled to buy a new video card capable of 16 million at this resolution. When fewer than 16M colors are available, Pasokon TV uses dithering to increase the perceived number of colors.