> the assumption that you know all of the tests that you will need before you know your software

I'm not a TDDer but must object on their behalf: not only is that not what they said, it couldn't be further from it! What they taught was to work in short iterations and work out a bit of requirements, design, and implementation in each one. If you needed knowledge you didn't have yet, they would advise you to make each cycle shorter until you did. Sometimes they'd take this to absurdity, breaking even simple features into design-test-implement cycles so microscopic that you would feel bored like a kid in class who can already see the next dozen steps and has to sit there waiting for the teacher to catch up.

The TDD approach was at the extreme tip of trying to get away from waterfall methods, so it would be quite some irony if it reduced back to that.

Edit: looks like I misread what taeric meant! But as McLuhan said, 'you don't like those ideas, i got others'...

In my experience there were two problems with TDD. One is that the micro-iterative approach amounted to a random walk in which features agglutinated into a software blob without any organizing design. Of course that's what most software development does anyway. But if you remember what Fred Brooks et. al. taught about having a coherent design vision for software that can organize a codebase at a higher level, the way iron filings are ordered by a magnetic field, that is a key anti-complexity vector which the TDD/XP/agile projects I saw all lacked. (Original XP had a notion of 'metaphor' that pointed vaguely in this direction but no one knew what it meant so it had no effect.) The lesson was that a coherent design vision doesn't evolve out of bit pieces, and there is an important place for the kind of thinking that got dismissed as 'top-down'. (Edit: by far the deepest work on integrating design coherence into TDD/XP approaches was Eric Evans' stuff on domain models and ubiquitous language. Eric is my friend but I knew he had the best stuff on this before that :)) (Edit 2: Eric points out that the people who created TDD were software design experts for whom coherent design was as natural as breathing air. The admonition against too much design thinking in TDD was intended for people like themselves, for whom the opposite mistake was never a risk, and didn't have unintended consequences until later.)

The other problem was that practices that started out as interesting experiments ossified into dogma so damn quickly that people started acting like they'd cracked the code of how to build software. A soft zealotry emerged that was deeply out of sync with the playful, adaptive spirit that good software work needs. This was unintentional but happened anyway, and there's a power law in how much creativity it kills: each generation that learns the Process is an order of magnitude less able to think for itself within it, and so must break free. I don't think the creators of the approach were any more at fault for this than the rest of us would have been in their shoes. The lesson is that this ossification is inevitable, even if you know all about the danger and are firmly resolved to avoid it. If there is any real breakthrough to be had at the 'how to build software' level, it is probably discovering some kind of antifreeze to put in there from the beginning to make sure this doesn't happen. Maybe there's a way to interrupt the brain and social circuitry that produce it. Failing that, I think that each project or maybe (maybe!) organization should hash out its own process from first principles and the prior experience of team members. That's ad hoc and involves a lot of repeating past mistakes for oneself but at least it isn't idiotifying. It's how I like to work, anyhow, and IMO that's what all software processes reduce to.