Although you might get a taste of the island over a weekend, I'd say you need at least five days to get a good grasp of what the island is all about. There are so many things to see in Crete that you might find you need more than a full week, or possibly even two.