So you have an easily identified problem to solve.

Specification and Responding to Change are not mutually exclusive. You do the first. You get an estimate. When a change is mandated, you specify the change. You estimate it. When you discover a feature was under-specified, you re-specify it.

You wouldn't find out your code doesn't work, and then not update the tests. Why would you treat specification any differently?

When the 3 month project ends up being a 9 month project, you have clear, actionable documentation on where your process needs fixing, and if the developer truly did hit all their marks, they get their well deserved rewards/kudos despite being saddled with a poorly managed project.

There are projects (I'm sure) that are practically impossible to specify well. I've never worked on one though. Web Development is, by it's nature, a rather straight-forward process. The more unpredictable portions (design, data-migration) can be sandboxed in a pragmatic manner.

I've been a part of projects that have run very well and satisfied the client despite changes doubling the original project cost. That's what Change Requests are for. To make sure you're all on the same page and everyone knows what they're paying for.

The fallacy of Agile for the projects I've worked on is that the clients will be delighted to recieve "what they want" if they don't know the ultimate timeline, can't make their own plans around it effectively, and have payed a lump sum of money thinking they were buying X, but instead recieved X--.

Since let's be realistic, when you talk about shifting to an "lean" process in a fixed-bid environment, you're not talking about delivering a better suited to the needs Y to a client who paid for X. You're talking about delivering a subset of X that fit within the time allowed. And who ever walked away happy when they got less than they paid for?

If you take for granted the Perfect Specification is impossible, you don't have to shoot the whole process in the head either. Would it be valuable to spend a few minutes post-design on deciding what Validations apply to the Form fields presented? Or how to map those fields to your data-store? Spend a few minutes thinking about what business-logic applies "in the background" once that Form is submitted? What happens on error? What test-cases should be defined to ensure we get the basics covered like HTML escaping, CSRF protection, etc?

Even a deliberate attempt to cover half of what you might ultimately need to consider to deliver this feature makes the rest so much easier, more predictable and easier to estimate.

The dirty secret is estimates (again, speaking from experience in Web Development) aren't all that difficult. It's just a Garbage-In-Garbage-Out situation. I've been doing this for 10 years. Estimating the time it'll take me to write a Form is not difficult to do accurately. Just like I can't off-the-top-of-my-head estimate the effort involved in writing a Web Framework or O/RM before breaking it down into Routers, Actions, Templates, etc, I can't estimate a Feature without breaking it down into smaller chunks either.

The overarching theme in most of the comments to this post is that estimates are hard, impossible or not worth doing. I respectfully disagree on all counts (within the context of web-development anyway, which is all I can speak to here).

It certainly doesn't benefit the Developer to have all the accountability and none of the authority when working on an extended project without strong Project Management backing him or her up. So it seems to me pretty self-defeating when instead of figuring out what works and what doesn't, like your average Developer would tackle any other problem, instead the popular opinion seems to veer towards throwing out the baby with the bath-water and absolving oneself of responsibility instead.

The Executives are going to blame someone, and it's rarely going to be themselves. At least not in any meaningful way. Like it or not if you're working on a Doomed Project, and you don't have someone to back you up, realistically you're going to be looking at a reevaluation in the capital of your own reputation. Fair or not, that's the way it works in my experience more often than not.