Probably because it's disconcerting to be upside-down and going backward, and it's harder to stay afloat. Both are easy to fix. First, get your bearings. Use a line of tiles or lights or other markings on the ceiling to help you set a straight course. Failing that, just hug the lane line.
Most pools have a set of colorful pennants hanging across the pool near each end wall.
To help stay afloat, learn to balance your body on your back. On your back, you keep your butt from sinking by leaning on your shoulder blades and the back of your head. That will keep your hips near the surface and you'll ride the waves like a pro, relaxing as you go. Inc. All rights reserved.