It is a game that is played by millions of people around the world. It can be played by young and old, fat and slim, and those in shape and out. Unfortunately, I don't think there's a whole lot of physical exertion going on at the local golf club. Every time I drive by a club I see plenty of less than fit human specimens making the rounds. In my books, if you can get off the couch after several years of inactivity and complete a round without breaking a sweat, it's a game.
Golf is a mind game more than anything else.
This places golf in the category of chess. I don't think that too many people would argue that chess is a sport. Give two people a chess set and tell them to walk through a meadow with it while playing and you've got golf. Why then is there such a push on to get golf into the sports category. By all accounts, it's already there.