This isn't advice per se, but rather a slightly different perspective.

The frustration and friction between technical staff and management staff that you describe is reality in every company that I have worked, from 50 employees to 50,000 employees. I hear that some new engineering-driven company cultures (Spotify, Zappos) are different, but I think those companies are extremely few and far between.

I am a software engineer by training and hobby, and a Director of IT by profession. One of my teams that is responsible for some Linux sys admin and some web programming is led by a very diligent and smart manager, who happens to have comparatively little technical knowledge. When I first started overseeing the team, there were occasional flair-ups between some engineers and the non-technical manager, which made both sides miserable. Since intervening a few times and employing a specific tactic, things have become noticeably better.

First, here’s what I have observed that leads to manager/engineer friction, and if not addressed, eventual misery.

Generally speaking and in my experience, engineers want to create things that are engineered beautifully: robust, elegant, simple, extensible, and easy to maintain. That takes time, patience, care, and hard work. To be asked to call a half-finished project good enough, or worse, to throw things out and consider all of the invested time as wasted, is naturally infuriating to anyone, but engineers are asked to do these two things more often than most other professions, in my experience.

Managers are paid to make sure what is being done is in the best interest of the organization, which has financial obligations to maintain. Many organizations are just squeaking by financially; they don't have the luxury of taking the care and patience that most engineers want to put in, and it’s managers’ jobs to find the right balance between good technical solutions and good business. The results of those decisions are messy, and rarely will management and the technical staff see things eye-to-eye.

These different perspectives and different motivations lead to friction, but poor communication exacerbates it. Managers have their own language, which has been parodied all over (drill down, actionable, win-win, ROI, etc.) and engineers have their own language (insert programming jargon and/or tech acronym soup here) that has been similarly parodied, and the two don’t overlap much.

When this happened on my team, I took a lot of time to talk to everyone involved about what was going on. Since I speak both manager-speak, and engineer-speak, I pretty quickly understood that what both were saying was reasonable. They just didn’t have a good understanding of what pressures and job-realities the other was dealing with, and didn’t have a common vocabulary to communicate their respective frustrations and challenges.

To smooth over the fraying relations on my team, I sat down with the engineering staff, and the manager, first separately, and then together. To the engineers I explained the goals and motivations of the manager, what pressure he/she was receiving from me, and what pressure I was receiving from the CIO and CEO. To the manager, I explained the realities that the technical team was dealing with – the occasional unknowns of software engineering, and how estimates can be dramatically off in ways that is not incompetence or malice, but merely unforeseen technical compilations.

Over time, I was able to teach the two factions to understand each other a little better. I asked them each to over-explain problems to one other, in simplified (shared) terminology, rather than throwing up their hands at what they thought were unreasonable demands or seemingly insubordinate responses. Over time, they have come to realize that the other group isn’t malicious or incompetent, but merely functions very differently from themselves. They trust each other a little more now, so when the engineers say something will take a week, the manager trusts them and doesn’t demand that it should be done in a day. When the manager explains that unfortunately we have to pull the plug on a project because our funding was cut, the engineers can trust that management genuinely fought for the project, but business reality dictated otherwise.

So if I have any advice for you, it is to get more involved in the business side of your organization. Business decisions and playing politics are rarely as stupid as they seem once you understand them.