Well, I was surprised to learn that there are various reasons for doing this. The main reason is that a double cap prevents their goggles from budging while swimming. They do this by putting on the first cap, and then their swim goggles and then placing the second cap atop it.
Another reason is that wearing two different material caps helps reduce body resistance while swimming. The first cap is usually a latex swim cap, which firmly sits on the head. The second cap is typically a cap silicone.