This is definitely a special place--just a bit inland, but well worth the effort for the beautiful location on a hill, with privacy, natural beauty, and breezes.
Once off the ferry, my son and I rented bikes and rode to Garden Gate. This trip alone was magical--we followed Google on a "road" that turned to no more than a grassy path winding past a farm that could have been in upstate Vermont, not the middle of the Atlantic!
The hosts made us feel like true guests: they were BI locals who graciously opened their house to us. And in the morning they cooked and served a lovely breakfast.
I'm a BI regular and have been going for years (and spent a few summers working there back in the day), so of course I highly recommend the island. But if you're planning a trip there, I highly recommend Garden Gate too!