Not at all!  I find it much easier flying a plane than driving a car.  There's no traffic to worry about, and plenty of sky to go around for everyone!  Landings are the tricky part, but with a little practise they become second nature and you'll wonder why you ever worried about it.