The best time to go is in the early morning (to beat the crowds) during the spring (for the most impressive flower displays).
To beat the crowds, the best time to go is in the early morning.