Because it works best with flat colours, it is most suitable for graphic images, type and interface elements. GIFs can also handle regular patterns very efficiently. A checkerboard pattern can be used to simulate a huge range of colours that are still web-safe. Where some graphic file formats that rely purely on run-length encoding do not compress such patterns at all, the GIF format does a very good job. The ability to reduce the bit depth of GIF images also helps to make files smaller.
Smaller files are faster files.
The JPEG file format uses a fairly complex set of algorithms which allow you to trade-off picture quality for file size. It will compress an image quite considerably before the effects of the compression start to show and works best on pictures that don't have large areas of flat colour like photographs and illustrations with lots of gradients and tonal changes. As a designer, you have to juggle image size, file format and compression techniques all the time to get the best result for the minimum bandwidth. Sometimes, it's not so clear and you have to try both to see which gives the most effective result. It is unfortunate that many designers who have moved from print to Web page design still think in print terms and try to force print layouts and styles on an alien medium.