Repository: asottile/all-repos

Issue:
gitolite source
Hi friend :)

I'd like to add a gitolite source and wanted to run the approach by you before sending a PR. A couple questions:

1. For getting the list of repositories, seems like the only way provided via the gitolite API itself is [the `info` command](http://gitolite.com/gitolite/user/#the-info-command). It provides a handy `-json` flag, e.g.:
   ```
   $ ssh git@git.mycompany.com info -json
   {
      "repos" : {
         "a_repo" : {
            "perms" : {
               "W" : 1,
               "R" : 1
            }
         },
         "some_repo-DEPRECATED" : {
            "perms" : {
               # "W" key is omitted entirely
               "R" : 1
            }
         }
      }
      "gitolite_version" : "1.2.3",
      "USER" : "git@somehost",
      "GL_USER" : "ckuehl",
      "git_version" : "1.2.3"
   }
   ```

   Potentially we could provide other ways to get this list (e.g. something like y/gits), but I'm thinking if I wanted to do that, I'd just format that into a JSON file and use the JSON file source.

   So, does over SSH sound okay? And if so, do you prefer subprocessing out to ssh or using paramiko? (I think I prefer the subprocess approach just to avoid adding a new dep?)

2. Actually I guess I only had one question. This looks pretty straightforward to implement :P

Comments:

Patch:
diff --git a/README.md b/README.md
index d81aa01..d0fd9e3 100644
--- a/README.md
+++ b/README.md
@@ -184,6 +184,29 @@ Clones all repositories available to a user on github.
 - `forks` (default `false`): whether to include repositories which are forks.
 - `private` (default `false`): whether to include private repositories.
 
+### `all_repos.source.gitolite`
+
+Clones all repositories available to a user on a
+[gitolite](http://gitolite.com/gitolite/index.html) host.
+
+#### Required `source_settings`
+
+- `username`: the user to SSH to the server as (usually `git`)
+- `hostname`: the hostname of your gitolite server (e.g. `git.mycompany.com`)
+
+The gitolite API is served over SSH.  It is assumed that when `all-repos-clone`
+is called, it's possible to make SSH connections with the username and hostname
+configured here in order to query that API.
+
+#### Optional `source_settings`
+
+- `mirror_path` (default `None`): an optional mirror to clone repositories from.
+  This is a Python format string, and can use the variable `repo_name`.
+
+  This can be anything git understands, such as another remote server (e.g.
+  `gitmirror.mycompany.com:{repo_name}`) or a local path (e.g.
+  `/gitolite/git/{repo_name}.git`).
+
 #### Directory location
 
 ```
diff --git a/all_repos/source/gitolite.py b/all_repos/source/gitolite.py
new file mode 100644
index 0000000..4cb9cc2
--- /dev/null
+++ b/all_repos/source/gitolite.py
@@ -0,0 +1,39 @@
+import collections
+import json
+import subprocess
+from typing import Dict
+from typing import List
+
+
+class Settings(collections.namedtuple(
+        'Settings', ('username', 'hostname', 'mirror_path'),
+)):
+
+    __slots__ = ()
+
+    def clone_url(self, repo_name):
+        return (
+            self.mirror_path or
+            f'{self.username}@{self.hostname}:{{repo_name}}'
+        ).format(repo_name=repo_name)
+
+
+Settings.__new__.__defaults__ = (None,)
+
+
+def _repo_names_from_source(settings: Settings) -> List[str]:
+    info = subprocess.check_output(
+        ('ssh', f'{settings.username}@{settings.hostname}', 'info', '-json'),
+    )
+    info = json.loads(info.decode('UTF-8'))
+    return set(info['repos'])
+
+
+def list_repos(settings: Settings) -> Dict[str, str]:
+    return {
+        # Repo names have ".git" appended to avoid naming conflicts between
+        # repos and directories in the gitolite hierarchy (a path could
+        # otherwise be both).
+        f'{repo_name}.git': settings.clone_url(repo_name)
+        for repo_name in _repo_names_from_source(settings)
+    }
diff --git a/tox.ini b/tox.ini
index 4b6f9a9..5487f6c 100644
--- a/tox.ini
+++ b/tox.ini
@@ -12,7 +12,7 @@ commands =
     pre-commit run --all-files
 
 [testenv:venv]
-envdir = venv-{[tox]project}
+envdir = venv
 commands =
 
 [pep8]


Test patch:
diff --git a/tests/source/gitolite_test.py b/tests/source/gitolite_test.py
new file mode 100644
index 0000000..a76d4a9
--- /dev/null
+++ b/tests/source/gitolite_test.py
@@ -0,0 +1,78 @@
+import json
+import subprocess
+from unittest import mock
+
+import pytest
+
+from all_repos.source import gitolite
+
+
+@pytest.fixture
+def settings():
+    return gitolite.Settings(
+        username='git',
+        hostname='git.mycompany.com',
+    )
+
+
+def test_clone_url_default(settings):
+    assert settings.clone_url('some_package') == (
+        'git@git.mycompany.com:some_package'
+    )
+    assert settings.clone_url('some/nested/package') == (
+        'git@git.mycompany.com:some/nested/package'
+    )
+
+
+def test_clone_url_custom_mirror_path(settings):
+    settings = settings._replace(
+        mirror_path='/gitolite/git/{repo_name}.git',
+    )
+    assert settings.clone_url('some_package') == (
+        '/gitolite/git/some_package.git'
+    )
+    assert settings.clone_url('some/nested/package') == (
+        '/gitolite/git/some/nested/package.git'
+    )
+
+
+@pytest.fixture
+def fake_info_response():
+    response = json.dumps({
+        'repos': {
+            'some_rw_repo': {
+                'perms': {'R': 1, 'W': 1},
+            },
+            'some_ro_repo': {
+                'perms': {'R': 1},
+            },
+        },
+        'gitolite_version': '1.2.3',
+        'USER': 'git@somehost',
+        'GL_USER': 'someuser',
+        'git_version': '1.2.3',
+    }).encode('UTF-8')
+
+    side_effect = {
+        ('ssh', 'git@git.mycompany.com', 'info', '-json'): response,
+    }.__getitem__
+
+    with mock.patch.object(
+            subprocess, 'check_output', side_effect=side_effect,
+    ):
+        yield
+
+
+@pytest.mark.usefixtures('fake_info_response')
+def test_repo_names_from_source(settings):
+    assert gitolite._repo_names_from_source(settings) == {
+        'some_rw_repo', 'some_ro_repo',
+    }
+
+
+@pytest.mark.usefixtures('fake_info_response')
+def test_list_repos(settings):
+    assert gitolite.list_repos(settings) == {
+        'some_rw_repo.git': 'git@git.mycompany.com:some_rw_repo',
+        'some_ro_repo.git': 'git@git.mycompany.com:some_ro_repo',
+    }