We recommend having at least a couple of days in Esperance. Keep in mind that if you're coming from Perth it's a very long drive so you may need longer to recover and be able to fully enjoy your trip without feeling rushed.