To leave the Abandoned Village, you will need to return to your boat and navigate your way back through the cave.