This might be difficult to do but can you explain your overall architecture design? What sorts of issues are you having? Where are the pain points?

Going from top to bottom:

> Our company builds a java openGL CAD/CAM application suite for windows desktops

If it's an application suite then, from my understanding, you'll be building a main set of libraries and then a set of tools that all use these libraries. Have you considered a hierarchical plugin design? Have a main application that starts and setups all of your main rendering and CAD/CAM magic. Then go from there to working out a simplest of APIs to what everything actually needs access too.

Your main application basically just manages UIs/drawing to an OpenGL port. From there you can load modules to do other things. If you abstract what is needed then each module should only need to define How a functionality is executed, not where and what a functionality should look like in the UI. For instance refactor your code to follow such a structure:

Master UI System (Exposes: "Options", "Renderables", "Views")
-> Drafting Plugin (Exposes: "Models", "Collision", "Faces")
-> CAM Plugin (Exposes: "Routing Paths")

Master UI does not need to know anything about Design Plugin and CAM Plugin.

Drafting Plugin needs to know about Master UI but nothing about CAM Plugin.

CAM Plugin needs to know about Master UI and Drafting Plugin.

That's what I would try and do if this was a new project but this isn't one and uprooting your entire (or even any recognizable percentage of your code base) is unreasonable.

> We have a couple tens of million LOCs, with ~50 projects and 1000s of packages

If you've got that many packages then you might want to find out what sort of abstractions are being used, not working correctly, and remove them/replace them with simpler solutions. How much of these packages are filled with Interfaces/Abstract Classes/Implementations of interfaces

> After ~10 years of neglection we need a strategy to increase the code quality (lots of dependencies, feature envying inheritance hierarchies, spaghetti code, similar problem are solved in myriad ways, all that jazz).

One at a time:

> lots of dependencies

Slowly replace dependencies by either abstracting features further, replacing with new standard library features, or by implementing other solutions to the same problems. Every dependency is an added layer of complexity in my book so it's best to avoid this as much as possible.

> feature envying inheritance hierarchies

This comes as a side effect of not knowing what a level of abstraction is actually meant to be doing. Have a team meeting and ask what each team thinks the actual problems that are needing to be solved are. The people knee deep in crap will have a better idea of what's the correct or natural abstraction for these cases if the ones currently being used are unnatural. It may just be that the code base has had too many large scale changes or even just have had too many features pushed in at once (which for a CAM/CAD tool is definitely not unheard of, this is a very specialized and hard task)

> spaghetti code

Get some sort of static analyzer. I remember one group I worked with used Sonar. Also remember that the best code quality tool is a good agreed upon set of standards. Somethings that have worked for me on some group projects I've worked on has been: Avoid complicated constructors, always default a variable to final, avoid complicated logic statements always exit early rather then filter before in a for loop, use all the up-to-date constructs to aid with code clarity (try(stream), for(var:set), and more).

> similar problem are solved in myriad ways

If there is one problem that exists in two places this is an opportunity for you to pull the part out, abstract it, and use it as a library. This is a double edged sword since these two parts actually need to contain the same problems which some times is not the case.

Now to the nitty gritty:

> How do you measure code quality? How do you interpret the metrics?

(How many times does the code result in an error) * (The time in hours that it takes to debug the code).

Larger number is worse. Keep a notebook/log of these times, graph them, and use that as a map to decide what is worth refactoring. If a piece of code "just works" but looks ugly it can wait to be refactored if there is another piece of code that looks "visually appealing" while still causing daily side effects in the active development of the project.

> What are good tools for a windows/java/eclipse dev environment?

I've always managed ANT scripts for my group projects since they are very very cross platform. Maven works great but I'm not a fan of the complexity of install for non-linux users. Also check out IntellJ for built-in maven support.

> How do you act on the metrics and actually improve code quality?

Change your code by coming at it from a different perspective. If that perspective yielded a more promising piece of code (that is easier to understand, causes less side effects, and uses less external/non-standard functionality) then you keep it. A lot of my code I write is code I throw away. This is much harder to justify to business people but it's an important part of the process to sketch up what you think might work even if the attempts aren't always fruitful.

> Can you recommend any resources of success stories on how companies managed to increase code quality of a big, tangled system?

Check out the U.S. Digital Service for the only recent success story that comes to mind [1].

If anyone knew the secret sauce they wouldn't give it out for free. The ability to "Fix" all the "Broken" projects isn't an issue on the scales that we think they are. A large portion of all technology-related projects fail [0]. If anyone could prove they where able to reliably fix these issues they'd be billionaires over night.

[0] - http://www.zdnet.com/article/study-68-percent-of-it-projects...

[1] - http://www.theatlantic.com/technology/archive/2015/07/the-se...

Edit: Removed "What do you mean by thousands of packages?"

Looking forward to what you think of all this.