Of course, there's also good old foot power. If you want to bring a bicycle on the train, you must contact the LIRR and get a permit. Only certain trains allow bicycles. Also, only service dogs are allowed on the LIRR, so if you want to bring Fido, you'll probably have to drive. The Hampton Jitney is a bus service that makes several stops in Manhattan and then several stops in the Hamptons.
Reservations are essential at peak times.
One added convenience is that the Jitney picks up from various locations on the Upper East Side of Manhattan and from locations near the airports. The best way to stay is to practice the fine art of mooching and stay with friends who have a house. Best of all, it's cheap. Other houses are rented out for the summer, usually at exorbitant rates. Sorry, but that's just not an option for most of us.