This comment ended up as a bit of a rant, on a topic where you have way more experience than I do. Sorry about that! At this point, I'm just using this as a chance to gather my own thoughts on the subject, and I figure I'll post it just in case I get some interesting responses.

-------------

Yeah, that's a great observation on Agile. Smaller but more accurate estimates + cheaper small course corrections.

The dual is that the rest of the organization needs to adapt too—good short-term estimates don't necessarily give you good long-term estimates! So we need loosely coupled, self-sufficient teams that don't depend on accurate estimates from other teams.

I've always hated formal Agile processes (especially Scrum) but I have to work with teams that use them, so I did a bunch of reading about it. The goals and even a lot of the concrete ideas behind it made a lot of sense! It's just that every single concrete instance of the process fell short. I've seen several concrete failure modes in practice:

1. Scrum-flavored micromanagement. People (not necessarily formal managers) use the process to dictate exactly what everyone else needs to work on at practically a daily granularity.

2. Siloing. People are incentivized to only work on "their" tasks—ad hoc collaboration with teammates, trying out speculative design ideas and experiments, figuring out better ways to do something, all get in the way of finishing your tickets, which makes you look flaky in inevitably awkward group meetings.

3. Loss of context. People see that the process helps the team switch priorities, so they switch focus every sprint. Couple this with micromanagement, and you get individuals moved around on wholly unrelated tasks. Too much context switching adds a lot of mental overhead and makes it much harder to build anything with a coherent design and vision (either from a product or from an engineering standpoint).

4. Performance mismeasurement. The whole process is supposed to help the team with estimates. Tracking metrics the process generates and treating them as a way to measure individual performance (whether formally, or just through peer pressure) doesn't work well and is awful for morale.

I haven't seen it, but I bet it is possible to have a Scrum process—or, more likely, something lighter like Kanban—that avoids these problems. But I believe that even idealized Agile processes put too much weight on consistency for its own sake. Making consistent incremental progress helps with estimates and, I suppose, makes the team somewhat easier to manage, but none of that matters nearly as much as, say, quality of work and impact!

Even if nothing from the Agile process makes it into formal performance evaluations, the ceremonies almost seem designed to create peer pressure around it. And that affects day-to-day job satisfaction a lot more than mere performance reviews. Although maybe this is another thing that goes away if you have high-trust teams that are legitimately self-organizing the way Agile is supposed to be... But I'm not sure even that would compensate for the pressure implicitly exerted by the structure of the process.

I bring this up because over-emphasizing consistency imposes a cost on people—individuals get a lot less flexibility in how they work, and end up feeling stress when they're less productive than normal. I tend to think an ideal environment should embrace inconsistency day-to-day and even week-to-week or month-to-month but that requires massive differences in culture. If a team pulls it off, the process starts mattering a lot less!