Random response order :)

> I would double and triple check your apps configuration, it seems there is something really unexpected going on.

More than one thing for sure. It's big and highly heterogenous ecosystem (multiple umbrellas) in distributed environment with high idempotency requirement. Without safeguards decisions were made that today make things very complicated. It's difficult to challenge long used patterns without hard recommendation or concrete evidence (I looked into Perceived Complexity analysis for Elixir but couldn't find anything).

My organization now builds the story of "Elixir codebase is hard to work and unreliable - let's switch to other tools". I don't like that story because I still remember all the fun I had and all the systems I produced that stood years with 0 maintenance. But those were small teams and small projects and today it's an enormous Jenga tower that's risky to breath around.

I would go as far as to say that our codebase is somewhat of a Petri dish for all kinds of issues (especially on dev/test envs, but not only). I've seen code merged to main branch because it wasn't picked up as changed and used stale cache, multiple-Elixir and OTP versions used in compilation, arch spillovers and more.

>I can see popular libraries spread across 3-4 different major versions), This is also something that should not be possible.

We have overrides and I don't see the umbrella test helper so I guess that umbrella-level overrides don't play nice with non-compiled in-app test runs.

> I am glad to say there is work happening towards this area (including soon the ability to search across all of the dependencies of your own project).

Looking forward to it, one thing that I often change too, is changelog across libraries, so it would be nice to always have those up-to-date.

> It should not be using one core, even it if compiles one dependency at a time. My whole point is that you get parallelism from within the dependency/project.

Project or a single-dependency compilation is fine - I felt different after recent updates to our stack and won't complain. In one umbrella I have opened I see ~250 deps packages and deps.tree shows me ~6500 lines of output, some of those are compiled multiple times - I blame the loops.

I have similar CPU - 8 performance cores and 4 efficiency ones. Usually deps.compile takes less than 100% of total CPU with spike to 150%. On partitioned tests I can feel the warmth of 1100% CPU usage (it also makes me smile, because I like big numbers). Right now I'm thinking that maybe I could spawn ~250 containers, make each compile dependency and then merge output into one and see what broke ;-)

> So you are getting the download count of one package, one that has been added to Erlang/OTP (and Elixir itself) and is more than expected to decrease in download count, to estimate the popularity of the whole language and a ecosystem of 20k+ packages?

Not ideal, but the best I could find. I also looked at Ecto which is standard, but figured out that json is more often used in projects than a database. Given quality of software itself I'd expect steady increase on the "core" libraries. But I also hear from prior projects about them being sunsetted. 2 or 3 projects in Elixir less, no big deal. In current organization few of us are actively advocating for Elixir and BEAM. We're minority and newcomers encounter as a first thing difficult stack setup (Erlang and Elixir version) long compilation time, hundreds of odd warnings and LSP that takes 40s to pick up on changes and highlight some errors.

--

I'm not in position of making any demands, it's self inflicted 99% of the time, and it's not a bug that can be fixed upstream it's just a subjective experience, and I wish it could be better.