Package: python-gitlab-submodule
python-gitlab-submodulev0.2.4List project submodules and get the commits they point to with python-gitlab.TheGitlab REST API V4doesn't implement anything for submodule inspection yet. The only thing we can
currently do isupdating a submodule to a new commit id,
but how can we trigger such an update if we don't know the current commit id
of our submodule?If you're usingpython-gitlaband you're distributing shared code among
your projects with submodules, you've probably run into this issue already.This package provides minimal utils to list the submodules present in a
Gitlab project, and more importantly to get the commits they're pointing to.Internally, it reads and parses the.gitmodulesfile at the root of the
Project. To get the commit id of a submodule, it finds the last commit that
updated the submodule and parses its diff (this can sometimes fail due to alimit of the GitLab API itself-
seeLimitations).About the future of this packageI don't plan to make PRs topython-gitlabfor now.In my opinion this problem should ideally be fixed in the Gitlab REST API,
and thenpython-gitlabcould wrap around the new endpoints.So I see this package as a temporary solution until the API gets extended
with more submodule functionalities.@darkdragon-001created an issue on
GitLab about the lack of support for submodules, feel free to support it with
a thumb up:https://gitlab.com/gitlab-org/gitlab/-/issues/352836RequirementsPython >=3.7(required bypython-gitlabsince version3.0.0)Dependenciespython-gitlabgiturlparseInstallpip install python-gitlab-submoduleor directly from Github:pip install git+git://github.com/ValentinFrancois/python-gitlab-submodule#egg=python-gitlab-submoduleUsage example1)Iterate over the submodules of the GitlabInkscapeprojectFor each submodule, print:its path in the projectits own Gitlab project SSH URLthe current commit sha that the Inkscape project is pointing tofromgitlabimportGitlabfromgitlab_submoduleimportiterate_subprojectsgl=Gitlab()inkscape=gl.projects.get('inkscape/inkscape')subprojects=iterate_subprojects(inkscape,gl,# current HEAD of master as I'm writing thisref='e371b2f826adcba316f2e64bbf2f697043373d0b')forsubprojectinsubprojects:print('-{}({}) ->{}'.format(subproject.submodule.path,subproject.project.web_url,subproject.commit.idifsubproject.commitelse'?'))Output:- share/extensions (git@gitlab.com:inkscape/extensions.git) -> 6c9b68507be427bffba23507bbaacf3f8a0f3752
- src/3rdparty/2geom (git@gitlab.com:inkscape/lib2geom.git) -> 9d38946b7d7a0486a4a75669008112d306309d9e
- share/themes (git@gitlab.com:inkscape/themes.git) -> 2fc6ece138323f905c9b475c3bcdef0d007eb2332)Iterate over the submodules of the GitlabInkscapeprojectFor each submodule, print:its path in the projectif its commit is up-to-date with the HEAD of the main branch of the
subprojectfor subproject in subprojects:-    print('- {} ({}) -> {}'.format(-        subproject.submodule.path,-        subproject.project.web_url,-        subproject.commit.id if subproject.commit else '?'))+    head_subproject_commit = subproject.project.commits.list(+        ref=subproject.project.default_branch)[0]+    if subproject.commit is None:  # can happen with very large commit diffs+        status = 'cannot check'+    elif subproject.commit.id == head_subproject_commit.id:+        status = 'ok'+    else:+        status = '/!\\ must update'+    print('- {}: {}'.format(subproject.submodule.path, status))Output:- share/extensions: /!\ must update
- src/3rdparty/2geom: ok
- share/themes: okAvailable functions and objectsiterate_subprojects(...)What you'll probably use most of the time.YieldsSubprojectobjects that describe the submodules.iterate_subprojects(project:Project,gl:Union[Gitlab,ProjectManager],ref:Optional[str]=None,only_gitlab_subprojects:bool=False,self_managed_gitlab_host:Optional[str]=None)->Generator[Subproject,None,None]Parameters:project: agitlab.v4.objects.Projectobjectgitlab: thegitlab.Gitlabinstance that you used to authenticate, or itsprojects: gitlab.v4.objects.ProjectManagerattributeref: (optional) a ref to a branch, commit, tag etc. Defaults to the
HEAD of the project default branch.only_gitlab_subprojects: (optional) if set toTrue, will ignore the
submodules not hosted on GitLab. If set toFalse(default), it will yieldSubprojectobjects withself.project = Nonefor submodules not hosted on GitLab.self_managed_gitlab_host: (optional) if some submodules are hosted on a
self-managed GitLab instance, you should pass its url here otherwise it
may be impossible to know from the URL that it's a GitLab project.Returns:Generator ofSubprojectobjectsLimitations:due tohttps://docs.gitlab.com/ee/development/diffs.html#diff-collection-limits,
some very large commit diffs won't be parsed entirely. This means that when
inspecting the diff of the latest commit that updated./<submodule_dir>,
in some rare cases./<submodule_dir>might not be part of the diff
object returned by the GitLab API. In that case we have no other choice than
setSubproject.committoNone, that's why the two examples above
check ifsubproject.commitis notNonebefore using the valuesubproject.commit.id.list_subprojects(...)Same parameters asiterate_subprojects(...)but
returns alistofSubprojectobjects.classSubprojectBasic objects that contain the info about a Gitlab subproject.Attributes:project: Optional[gitlab.v4.objects.Project]: the Gitlab project that the
submodule links to (can beNoneif the submodule is not hosted on GitLab)submodule:Submodule: a basic object that contains
the info found in the.gitmodulesfile (name, path, url).commit: Union[gitlab.v4.objects.ProjectCommit, Commit]: the commit that
the submodule points to (if the submodule is not hosted on GitLab, it will
be a dummyCommitobject with a single attributeid)Examplestr()output:<class 'Subproject'> => {
    'submodule': <class 'Submodule'> => {'name': 'share/extensions', 'parent_project': <class 'gitlab.v4.objects.projects.Project'> => {'id': 3472737, 'description': 'Inkscape vector image editor', 'name': 'inkscape', 'name_with_namespace': 'Inkscape / inkscape', 'path': 'inkscape', 'path_with_namespace': 'inkscape/inkscape', 'created_at': '2017-06-09T14:16:35.615Z', 'default_branch': 'master', 'tag_list': [], 'topics': [], 'ssh_url_to_repo': 'git@gitlab.com:inkscape/inkscape.git', 'http_url_to_repo': 'https://gitlab.com/inkscape/inkscape.git', 'web_url': 'https://gitlab.com/inkscape/inkscape', 'readme_url': 'https://gitlab.com/inkscape/inkscape/-/blob/master/README.md', 'avatar_url': 'https://gitlab.com/uploads/-/system/project/avatar/3472737/inkscape.png', 'forks_count': 900, 'star_count': 2512, 'last_activity_at': '2022-01-29T23:45:49.894Z', 'namespace': {'id': 470642, 'name': 'Inkscape', 'path': 'inkscape', 'kind': 'group', 'full_path': 'inkscape', 'parent_id': None, 'avatar_url': '/uploads/-/system/group/avatar/470642/inkscape.png', 'web_url': 'https://gitlab.com/groups/inkscape'}}, 'parent_ref': 'e371b2f826adcba316f2e64bbf2f697043373d0b', 'path': 'share/extensions', 'url': 'https://gitlab.com/inkscape/extensions.git'},
    'project': <class 'gitlab.v4.objects.projects.Project'> => {'id': 5833962, 'description': 'Python extensions for Inkscape core, separated out from main repository.', 'name': 'extensions', 'name_with_namespace': 'Inkscape / extensions', 'path': 'extensions', 'path_with_namespace': 'inkscape/extensions', 'created_at': '2018-03-22T00:29:09.053Z', 'default_branch': 'master', 'tag_list': ['addin', 'additional', 'addon', 'core', 'extension', 'inkscape', 'python'], 'topics': ['addin', 'additional', 'addon', 'core', 'extension', 'inkscape', 'python'], 'ssh_url_to_repo': 'git@gitlab.com:inkscape/extensions.git', 'http_url_to_repo': 'https://gitlab.com/inkscape/extensions.git', 'web_url': 'https://gitlab.com/inkscape/extensions', 'readme_url': 'https://gitlab.com/inkscape/extensions/-/blob/master/README.md', 'avatar_url': 'https://gitlab.com/uploads/-/system/project/avatar/5833962/addons.png', 'forks_count': 89, 'star_count': 41, 'last_activity_at': '2022-01-29T19:10:13.502Z', 'namespace': {'id': 470642, 'name': 'Inkscape', 'path': 'inkscape', 'kind': 'group', 'full_path': 'inkscape', 'parent_id': None, 'avatar_url': '/uploads/-/system/group/avatar/470642/inkscape.png', 'web_url': 'https://gitlab.com/groups/inkscape'}},
    'commit': <class 'gitlab.v4.objects.commits.ProjectCommit'> => {'id': '6c9b68507be427bffba23507bbaacf3f8a0f3752', 'short_id': '6c9b6850', 'created_at': '2021-11-28T22:23:47.000+00:00', 'parent_ids': ['fdda3f18b3ddda61a19f5046ce21a6e2147791f5', '8769b39a55f94d42ac0d9b24757540a88f2865cc'], 'title': "Merge branch 'add-issue-template-bug-report' into 'master'", 'message': "Merge branch 'add-issue-template-bug-report' into 'master'\n\nadd issue template for GitLab for bug reports\n\nSee merge request inkscape/extensions!377", 'author_name': 'Martin Owens', 'author_email': 'doctormo@geek-2.com', 'authored_date': '2021-11-28T22:23:47.000+00:00', 'committer_name': 'Martin Owens', 'committer_email': 'doctormo@geek-2.com', 'committed_date': '2021-11-28T22:23:47.000+00:00', 'trailers': {}, 'web_url': 'https://gitlab.com/inkscape/extensions/-/commit/6c9b68507be427bffba23507bbaacf3f8a0f3752', 'stats': {'additions': 25, 'deletions': 0, 'total': 25}, 'status': 'success', 'project_id': 5833962, 'last_pipeline': {'id': 417958828, 'iid': 924, 'project_id': 5833962, 'sha': '6c9b68507be427bffba23507bbaacf3f8a0f3752', 'ref': 'master', 'status': 'success', 'source': 'push', 'created_at': '2021-11-28T22:23:48.313Z', 'updated_at': '2021-11-28T22:31:49.083Z', 'web_url': 'https://gitlab.com/inkscape/extensions/-/pipelines/417958828'}, 'is_exact': True}
}list_submodules(...)Lists the info about the project submodules found in the.gitmodulesfile.list_project_submodules(project:Project,ref:Optional[str]=None)->List[Submodule]Parameters:project: agitlab.v4.objects.Projectobjectref: (optional) a ref to a branch, commit, tag etc. Defaults to the
HEAD of the project default branch.Returns:listofSubmoduleobjectsclassSubmoduleRepresents the.gitmodulesconfig of a submodule + adds info about the
parent projectAttributes:parent_project: gitlab.v4.objects.Project: project that uses the submoduleparent_ref: str: ref where the.gitmodulesfile was readname: str: local name used by git for the submodulepath: str: local path pointing to the submodule directory in the projecturl: str: URL linking to the location of the repo of the submodule (not
necessarily Gitlab)Examplestr()output:<class 'Submodule'> => {'name': 'share/extensions', 'parent_project': <class 'gitlab.v4.objects.projects.Project'> => {'id': 3472737, 'description': 'Inkscape vector image editor', 'name': 'inkscape', 'name_with_namespace': 'Inkscape / inkscape', 'path': 'inkscape', 'path_with_namespace': 'inkscape/inkscape', 'created_at': '2017-06-09T14:16:35.615Z', 'default_branch': 'master', 'tag_list': [], 'topics': [], 'ssh_url_to_repo': 'git@gitlab.com:inkscape/inkscape.git', 'http_url_to_repo': 'https://gitlab.com/inkscape/inkscape.git', 'web_url': 'https://gitlab.com/inkscape/inkscape', 'readme_url': 'https://gitlab.com/inkscape/inkscape/-/blob/master/README.md', 'avatar_url': 'https://gitlab.com/uploads/-/system/project/avatar/3472737/inkscape.png', 'forks_count': 900, 'star_count': 2512, 'last_activity_at': '2022-01-29T23:45:49.894Z', 'namespace': {'id': 470642, 'name': 'Inkscape', 'path': 'inkscape', 'kind': 'group', 'full_path': 'inkscape', 'parent_id': None, 'avatar_url': '/uploads/-/system/group/avatar/470642/inkscape.png', 'web_url': 'https://gitlab.com/groups/inkscape'}}, 'parent_ref': 'e371b2f826adcba316f2e64bbf2f697043373d0b', 'path': 'share/extensions', 'url': 'https://gitlab.com/inkscape/extensions.git'}submodule_to_subproject(...)Converts aSubmoduleobject to aSubprojectobject, assuming it's
hosted on Gitlab.Raises aFileNotFoundErrorif the path of the submodule actually doesn't
exist in the host repo or if the url of the submodule doesn't link to an
existing repo (both can happen if you modify the.gitmodulesfile without
using one of thegit submodulecommands).submodule_to_subproject(gitmodules_submodule:Submodule,gl:Union[Gitlab,ProjectManager],self_managed_gitlab_host:Optional[str]=None,)->SubprojectParameter details: Seeiterate_subprojects(...)