It's worth it. Use a screwdriver to remove anything unpainted that should stay that way. Typically, that's switchplates and various other hardware. Drop the covers upside down onto the floor underneath your dropcloth. Go around the entire room and closely examine any cracks or holes and do some hefty gouging to loosen brittle areas.
Some people try to do two coats of the finish paint.
Better to use primer, as it's designed to cover and seal well enough so the finish coat will glide on smoothly. Do one wall at a time. Do the walls first, and save the windows, doors and trim for after the walls dry. If you're doing the ceiling, always do that first since paint will splatter onto the walls. Cut-in with a brush all the areas that a roller can't reach.