Currently under construction, probably through the fall months, so it is a bit messy like all construction areas. I can't say it's the same for flights to other areas. You may want to individually check this out. This allows for better connections to your destination. Trip is longer, about 2-2.5 hours on busses.
Airline tickets are cheaper.
While the map shows train tracks going through West Point, they are for freight trains only. The passenger service is on the east side of the Hudson at Cold Spring. The train leaves from Grand Central in the heart of Manhattan and you would have to take a taxi at each end of the journey. Basically, renting a car is recommended. The convenience and peace of mind should be well worthwhile.