> I want feature branches to be short lived and integration to happen (predominantly) via master, because that offers significant benefits. For the cases where I really needed something from A in B, rebasing or cherry-picking has not been much of a hassle for me in practice.

I think having everything close to master offers benefits, but when it means people start doing things like feature flags then that's a higher cost than the benefits, so feature branches need to live long enough to implement a feature enough to tell whether it works - which is probably a week or two even in the best cases. Going from "everything other than master is private, you ignore what other people are working on until it hits master" to "all pushed feature branches are public unless otherwise stated, you pay attention to what other people are doing and pull their branches whenever you think that would be helpful" is a significant change in mentality, but I found it to be the biggest benefit of moving to DVCS - perhaps because resolving conflicts is disproportionately frustrating work, IME.

> you need to to record which commits have passed CI in some way that's trivial to use for bisect skip (e.g. by CI rewriting the head commit's message to indicate it was tested and passed). This is pretty easy to set up and very useful.

In my experience you don't need full CI, because people mostly test the code they've just written at least a little bit. Even if they broke something in one commit, they'll usually fix it in the next.

> But you can have the exact same thing happen with a merge based workflow, firstly because you can never prevent someone from (badly) rebasing master into their private local branch

You can't prevent someone who's determined to put broken commits in their history from doing so, sure. But you can ask everyone to not use rebase and they'll do it, and you can ask them to only commit code that compiles (or passes tests) and they'll do that most of the time (and if you have the occasional isolated non-compiling commit, that's not such a big problem).

> secondly because it is also quite likely that someone who couldn't be asked to rebased cleanly but uses merges exclusively would merge master multiplie times into their branch without making the merge commits necessarily pass tests, instead they'd probably also concentrate their efforts on making the head of the feature branch pass.

The head of the feature branch is the merge commit, that's the whole point. They can't do any more work on their feature until they've at least made things compile - the first time they try to go through the edit-test cycle, they have to fix anything that they broke when merging. So sometimes a lazy person will make a non-compiling merge and then fix it in the following commit, but that's not too bad because it only leaves a single isolated non-compiling commit in the history.

> Kinda. I think the correct way to handle this is make it painful and requite people to duplicate the code in repo. You sometimes need two versions of X in production in parallel for a longer amount of time (e.g. if you are transitioning to a new architecture, and you need to run it as a shadow system for some time first to gather confidence), but it should not be a common thing and I find having both in the working tree works well for these cases.

I'm talking about the case where X and Y are separate systems; if your repository has "the current version of X" and "the current version of Y" then that can end up being pretty misleading because what you don't have in your repo is "what X thinks is the current version of Y" i.e. the version of the interface/client code to Y that X is using.

> That's a big no-no in my book. Maybe there is some scale at which this is the lesser evil because otherwise you make it too painful to refactor stuff, but in general I think you should make it painful for people to do this: all versions should be in master and there should be strong pressure to avoid having more than one version of anything in master, certainly for anything but a short transition periods.

If the library is shared between two systems that are versioned/released/deployed separately then it's necessarily normal to have multiple different versions of the library in production at the same time. IMO at that point it's best for the library to have its own proper release cycle in its own repository and the two systems to have their own repositories, and then each repo's history is an accurate reflection of the thing it's a repository for, but you're prevented from trying to view a combined history of the library and the systems that use it in a naive way, because such a history would always be misleading.

If the library's users are all the same team (i.e. part of the same standup etc.) - then it's usually better to treat everything that uses it as a single system and deploy it all at the same time, and then you can keep them all in the same repository and have a single consistent history. But if the library is shared by independent teams then you can't enforce that versioning and deployment are done together.

> There are massive downsides to allowing people to use random versions of internal libraries: security concerns, people ending up on some completely outdated version of a lib that then suddenly breaks completely for them and they don't have the time budget to rewrite all their crap, the massive cost of not being able to look into a single repo and know that if you see "import foolib" it's the foolib in the same source tree.

> I don't think it's necessary for master to correspond 100% to production, but it should be quite close. If you land something in master and it's not shipped by the end of the day, I'd say that's generally a bad thing and you should consider reverting.

All that's true; in the scenario where you're sharing an internal library between multiple systems you actually need to treat that library as a first-class project with semver, release notes, a security update/LTS policy and all the rest of it. But as soon as your organisation gets too big to deploy everything every time someone wants to release a feature, what do you do? It's now inevitable that you will have different versions of foolib running in production, so IMO the best thing is to have a repo structure that reflects that. I don't think you're actually contradicting what I've said: one repo for each system that's versioned and deployed together, if that system is your whole organisation and your whole codebase then great.

> You found a problem in production and would like to see where it came from: if production is either a monorepo or a "super-repo" composed of subrepos, you can just bisect. You can just git log the whole thing to understand recent history etc. You can git grep and it will show you all the uses of something, as opposed you have to figure out how to navigate dozens of different things done by different teams.

At the point where you decouple the versioning/releasing of different components, the things you want your tools to do become different. You can't check if a method is unused by grepping, because there may be an older version of an intermediate library that's using it. You don't want to bisect in the details of another team's changes, so bisect landing you on either the point where your commit broke or the point where you commit an upgrade of the library that you're using and that broke things is fine, probably better than landing on an internal commit in that library. Even if you do want to do the fix in that library yourself, coming up with a test case for it that's specific to the library rather than using the rest of your system is something you need to do anyway, because that library's test suite should be self-contained. The way of working is different enough that I don't want to use the same interface to it.