You cannot harvest eggplants too early. Baby eggplants are a treat to eat. However, it would be best if you did not leave them for too long—harvest eggplant when it is one-third to two-thirds full mature size.