You can travel to Fontaine as soon as you've completed the Archon Quest named The Outlander Who Caught the Wind, but it's best to wait until you've at least finished Song of the Dragon and Freedom, as this will unlock a Teleport Waypoint on your map, just north-west of Sumeru.
Once you've arrived, watch the short cutscene and then glide your way down towards the ocean. There's a Waverider summon point you can find here to sail to the Romaritime Harbor. Don't forget to explore some of the surrounding boats for collectibles.
Take the staircase up to the lift, and then travel up to the Statue of the Seven. Activate it to unlock part of Fontaine's map, and then return back on the other side of the lift. The Aquabus should be waiting for you, but you can summon it if it's not.
Ride the Aquabus with Aeval and enjoy the tour. You'll eventually arrive at the Court of Fontaine. Take the lift up to the fourth floor and you'll spot another Statue of the Seven just on the path off to your left. Activate it and enjoy all that Fontaine has to offer.