Clue: Immediately when Derby, say, fell (2,3,4,2,1,3)
Answer: AT THE DROP OF A HAT
Explanation: referring to a Derby as a type of hat