jenkinsci/git-plugin — pull request #213 by gloeglm (2014-03-20 16:16:12+00:00)
Title: Do polling using a shared repository with a single fetch

Okay so this is a fairly experimental change we are running on our local jenkins installation currently. It changes the git polling mechanism to no longer do a fetch on the workspace repository and do a check for changes on that, but to do only a single fetch on a central, shared clone of the remote and use that for change detection.

The motivation for this is that due to past decissions we are currently maintaining a jenkins setup with over 200 build jobs all building from the same git repo. So every push to the repo triggered polling for all those jobs, meaning 200+ concurrent git fetch commands. This caused huge IO load on our build servers, which lead to it sometimes taking 15+ minutes between a push and a project starting to build.

So with this change, there is only a single fetch being done, on a central repository (one per git url) that can then be used by all projects for detecting changes. For us this cut the time to check all projects for changes down to ~10 sec, obviously a huge improvement.

So this is all a little bit unknown territory for us, so we would appreciate any advice on how to improve this and feedback if this is something that would be possible to get merged. We would be more than happy to work on getting this to a merged state.