I highly recommend Hamming's response, which was very influential to the way I think about mathematics and physics:

https://www.dartmouth.edu/~matc/MathDrama/reading/Hamming.ht...

Edit: a teaser:

"Let us next consider Galileo. Not too long ago I was trying to put myself in Galileo's shoes, as it were, so that I might feel how he came to discover the law of falling bodies. I try to do this kind of thing so that I can learn to think like the masters did-I deliberately try to think as they might have done.

Well, Galileo was a well-educated man and a master of scholastic arguments. He well knew how to argue the number of angels on the head of a pin, how to argue both sides of any question. He was trained in these arts far better than any of us these days. I picture him sitting one day with a light and a heavy ball, one in each hand, and tossing them gently. He says, hefting them, "It is obvious to anyone that heavy objects fall faster than light ones-and, anyway, Aristotle says so." "But suppose," he says to himself, having that kind of a mind, "that in falling the body broke into two pieces. Of course the two pieces would immediately slow down to their appropriate speeds. But suppose further that one piece happened to touch the other one. Would they now be one piece and both speed up? Suppose I tied the two pieces together. How tightly must I do it to make them one piece? A light string? A rope? Glue? When are two pieces one?"

The more he thought about it-and the more you think about it-the more unreasonable becomes the question of when two bodies are one. There is simply no reasonable answer to the question of how a body knows how heavy it is-if it is one piece, or two, or many. Since falling bodies do something, the only possible thing is that they all fall at the same speed-unless interfered with by other forces. There's nothing else they can do. He may have later made some experiments, but I strongly suspect that something like what I imagined actually happened. I later found a similar story in a book by Polya [7. G. Polya, Mathematical Methods in Science, MAA, 1963, pp. 83-85.]. Galileo found his law not by experimenting but by simple, plain thinking, by scholastic reasoning.

I know that the textbooks often present the falling body law as an experimental observation; I am claiming that it is a logical law, a consequence of how we tend to think."

[..]

"But if you do not like these two examples, let me turn to the most highly touted law of recent times, the uncertainty principle. It happens that recently I became involved in writing a book on Digital Filters [8. R. W. Hamming, Digital Filters, Prentice-Hall, Englewood Cliffs, NJ., 1977.] when I knew very little about the topic. As a result I early asked the question, "Why should I do all the analysis in terms of Fourier integrals? Why are they the natural tools for the problem?" I soon found out, as many of you already know, that the eigenfunctions of translation are the complex exponentials. If you want time invariance, and certainly physicists and engineers do (so that an experiment done today or tomorrow will give the same results), then you are led to these functions. Similarly, if you believe in linearity then they are again the eigenfunctions. In quantum mechanics the quantum states are absolutely additive; they are not just a convenient linear approximation. Thus the trigonometric functions are the eigenfunctions one needs in both digital filter theory and quantum mechanics, to name but two places.

Now when you use these eigenfunctions you are naturally led to representing various functions, first as a countable number and then as a non-countable number of them-namely, the Fourier series and the Fourier integral. Well, it is a theorem in the theory of Fourier integrals that the variability of the function multiplied by the variability of its transform exceeds a fixed constant, in one notation l/2pi. This says to me that in any linear, time invariant system you must find an uncertainty principle."