by ninetyninenine | 2025-04-14 14:59:58+00:00
> That questions: Why are you allowing your programs to be deployed when tests are failing? This is not a realistic scenario in the real world. Yes, you can invent contrived hypotheticals all day long, but it is meaningless. We've been clear that we are referring to practical settings.

Tests don’t catch everything. You can have a billion tests and there can still be uncaught runtime errors.

If you had a language that probably does not have runtime errors you don’t even need one test. Your program cannot fail in that way.

I honestly don’t think you know what you’re talking about. I didn’t make up a single hypothetical. This is real. Production errors can happen in spite of tests. Are you not familiar with this happening? It just means this: no experience.

Your compiler having a bug or not is orthogonal to the topic. Again you don’t know what you are talking about. If our compiler allows for runtime errors but is fully correct then no amount of tests can guarantee a runtime error will never happen. Golang as a fully correct compiler cannot be gauranteed to have no runtime errors with tests ever.

> What you are referring to is clear, but it cannot be considered in a vacuum. The alternative is to see the program keep trodding along, but do the wrong thing. In that case who cares if the program crashes instead? You're getting incorrect behaviour either way.

You’re writing this because you don’t have experience with programs that can never crash. A program that doesn’t crash doesn’t mean you never exit the program. The program can exit if you want it to. You just need to deliberately tell the program to exit. In golang if you do a division by zero, the program crashes. If you had sum types all divisions return an optional. Both paths of the optional must be handled by exhaustive matching so you must handle the case where the division yields a number or its undefined. If you want the program to exit when it is undefined you can do so. In golang the compiler doesn’t force you to handle both outcomes It just crashes. It’s the same with out of bounds access of an array.

Again real world testing doesn’t guarantee shit. A “complete type system” can be as extensive as dependent types like COQ or much simpler like rusts where you just have sum types and exhaustive pattern matching.

> Perhaps you can choose an example of where you wrote code in Go that produced a runtime error, show us your tests, and explain how the condition evaded your checks and balances?

Oh easy. we had a function that calculates velocity from a stream of input data. That’s (p2 - p1 / t2 - t1). Our integration tests and unit tests have dozens of tests that never yielded an error and we never saw an error in production for years. We switched to a new iot device that sometimes sent identical measurements to our system. Division by zero. We had a crash in production.

> I'm fascinated to learn how your code ran perfectly while under test but then blew up in production.

You’re inexperienced that’s why you’re fascinated. If you have formula involving velocity there an almost infinite amount of combinations that will never produce a runtime error and an infinite amount of parameter combinations that do. True full coverage that completely proves the function works with tests involves infinite tests. Better to prove the function works via proof with a simple extension to the type system. Sum types.