Repository: StackStorm/st2

Issue:
Efficient pack development at scale
##### SUMMARY

Currently stackstorm-exchange packs have a development workflow that requires the version to get bumped with every pull request.  As part of a discussion in a pull request for the vSphere pack, some limitations of the current methodologies came up which will hinder scaling development.  As the number of developers in the community grows, packs will need a better way to accumulate many changes between releases.

##### ISSUE TYPE
 - Best Practices

##### OS / ENVIRONMENT / INSTALL METHOD
StackStorm-Exchange

##### DISCUSSION
In the vSphere pack (and likely other packs) today, every pull request requires a version bump.  If two changes are committed to master and the version is not bumped, one cannot determine which version that they might have, since the master branch always has the version of the latest release in it, and there is no staging branch.

This will not scale, and a solution is needed to enable for expected growth in the community.  The boost community for example solves this by maintaining a master branch which is always deemed to be the latest release, and a "develop" branch to which all pull requests are targeted.  This allows each boost library to accumulate multiple changes in a holding area ("develop") until the maintainer deems them sound, good, etc.  The maintainer then merges "develop" into "master" (this should be a simple fast-forward), and then the maintainer updates the version and performs release activities (a final CI build, tagging the release, and bumping any version info to whatever will be next.)

To translate this mechanism to a pack repository, one would:

1. Create a "develop" branch off "master".
2. Change github to direct all PRs to "develop".
3. Now multiple developers can submit as many changes as they need.
4. Maintainer decides it is time for a release, and fast-forwards master to develop, then commits a change to pack.yaml with the version bump, then tags the release and performs any other release related activities.

This has many benefits:

1. Allows a pack to accumulate multiple changes between releases.
2. Avoids collisions on pack.yaml if every PR would normally need to update it with a version bump.
3. Puts the decision of what is a patch/minor/major bump on the maintainer, not the submitter.


Discussion:
This approach could work, and does for a lot of other projects. On the ST2 main repo we've opted for a development `master` and release branches for specific releases so we can maintain multiple versions of ST2 at the same time if needed. (For instance pushing fixes for our paying customers for previous versions). We would likely take that approach on an exchange pack once the active development got to a level that required it. In the mean time if you'd like to coordinate releases with others I see no reason why a development branch couldn't be contributed to by various users to batch changes in a release. So if you'd like to take that approach with coordinated individuals that would like to contribute to the pack I would support that. To date we just haven't seen enough active development on the pack to warrant the modified release process.
If the standard behavior for pack installation always used a tag called "latest" or "stable" then leaving master open as the collection of all new, unreleased changes seems reasonable.  One could install a pack in a variety of stable ways to get what you want:
#### Today
```
# st2 pack install vsphere
    installs master
# st2 pack install vsphere=v0.7.7
    installs a specific release
```
#### Proposed
```
# st2 pack install vsphere
    defaults to installing the tag "stable" or "latest"
    fails if the repository has no stable tag,
      encourages opening an issue,
      and provides instructions to work around it
# st2 pack install vsphere=master
    installs the current development tip (you have to request possible instability)
# st2 pack install vsphere=v0.7.6
    installs a specific version
```
Pack maintainers would then be able to allow for pull requests on master without version bumps in `pack.yaml`, and control the release cycle including when a release is promoted to being stable.  This would allow a pack to have a beta or testing tag that may be a fresh release, for example.  This avoids the dual branch scenario.  If release branches become necessary they would fit right into this strategy.  Part of the release cycle might be to bump `pack.yaml` in master to something like `v0.8.0-dev` after making a release , while at release time to change it to `v0.7.8`, then tag it that way.  The `-dev` designation would let people know they are not using a release.
 ```
# st2 pack install vsphere
    installs master
```
I just checked, and you're right. This now sounds unexpected to me, too.

I think the improvement is much easier.
Instead of installing `master` on `st2 pack install vsphere`  we should install from latest stable tagged release like `vX.Y.Z`, which is `v0.7.7` in case of `vsphere` pack https://github.com/StackStorm-Exchange/stackstorm-vsphere/releases.

This will leave enough space for maneuver in `master` branch for someone maintaining the pack. Additionally, it doesn't enforce overcomplication like `-dev` and `develop`: pack contributors can ask to version every PR or ship releases with a bigger amount of changes when version is not bumped in a PR.
That would be also consistent with our model  where we consider `master` as something "unstable" and git tag/release as stable between the majority of StackStorm repositories.

@jeking3 Do you think that would be a decent solution?

I'm surprised this doesn't happen and I was sure that we install from latest stable git tag/release in `st2 pack ...` by default, not from `master`. It's a good catch @jeking3!
Yes, that sounds reasonable.  This will solve the issues described.  Your suggestion further reduces tagging requirements on the maintainer by having st2 pick the most recent semver tag when one is not specified.
Agreed: `st2 pack install` should default to installing most recent tagged version, rather than installing from master.

New PRs can continue to be submitted & merged into master branch, *but* this will not automatically create new tag. This should only be done as part of a 'release' step that will incorporate bumping version in pack.yaml, and creating a new tag. The release process can then combine multiple PRs into one versioned release.

Needs:
 - Changes to pack install code
 - CI/CD process updates for Exchange to change how/when tags are created

Will also need some thought around:
1/ Installing from a `file:///` location - this is a git repo. What is expected behavior here? Install from tag, or master?
2/ Users with their own internal Exchange index - what if they are not tagging their releases? How do we handle things if there is only master, no version tags? May need to have some methods of enforcing old-style "always install from master"
> New PRs can continue to be submitted & merged into master branch, but this will not automatically create new tag. This should only be done as part of a 'release' step that will incorporate bumping version in pack.yaml, and creating a new tag. The release process can then combine multiple PRs into one versioned release.

I think current release automation for StackStorm Exchange is flexible enough as is.

At the moment, only pushing to `master` with version change in `pack.yaml` will enforce new git tag/release creation via automation.
This way pack maintainers (who really need it) have a chance to ask contributors (or commit to a [forked PR directly](https://help.github.com/articles/committing-changes-to-a-pull-request-branch-created-from-a-fork/)) and decide whether to change _or_ not to change pack version which will trigger a new release. It means someone who needs it is already covered and we don't change the behavior for everyone else.

And so I think the improvement is scoped mostly to st2 core.
> At the moment, only pushing to master with version change in pack.yaml will enforce new git tag/release creation via automation.

I'm not sure about that - I believe it always tags code based on a merge to master, it doesn't check that the PR includes pack.yaml
Quick example: https://github.com/StackStorm-Exchange/stackstorm-ansible/pull/27
Exchange automation is smart enough to detect version change in `pack.yaml` and use that number for git tag/release and so for example it's possible to bump a major version `0.5.0` -> `1.0.0` via pack.yaml.
It would be nice if the master on each pack's repository uses a [SemVer ](https://semver.org/) compliant version with a development tag of "-dev", for example:
```
version: v0.8.0-dev
```
The reasoning for this is that if someone specifically asks for `=master` then you want the pack version to specify `vx.x.x-dev` to signify that it is not a release.  Without this workflow it will look like a specific version is installed, when it is that version plus all the changes accumulated in `master`.

The release of a pack would then require two steps:

1. The maintainer commits a pack.yaml without a development tag.  CI automation detects this and treats it as a release.  CI would fail if a tag already exists with this version, so as to avoid human error.  This makes a release out of master, where the CI automation creates a tag matching what's in pack.yaml.  I'm not sure how github releases are inserted, but this probably happens here as well.
2. The maintainer commits another pack.yaml with a development tag.  CI automation ignores this.  Pulls from master once again look like development builds.


You can use that format if you want manually via `pack.yaml`, semver-compliant version would be accepted and created as dedicated git tag. No reason to ignore it and we're not committed to change the automation for this specific use case.

---

Relying on latest git tag instead of `master` in `st2 pack install` makes sense, - thanks for raising it! :+1: 
This change may be incorporated in a future st2 release.

Resolving patch:
diff --git a/st2common/st2common/util/pack_management.py b/st2common/st2common/util/pack_management.py
--- a/st2common/st2common/util/pack_management.py
+++ b/st2common/st2common/util/pack_management.py
@@ -370,7 +370,7 @@ def get_repo_url(pack, proxy_config=None):
         if not pack:
             raise Exception('No record of the "%s" pack in the index.' % (name_or_url))
 
-        return (pack['repo_url'], version)
+        return (pack['repo_url'], version or pack['version'])
     else:
         return (eval_repo_url(name_or_url), version)
 


Test patch:
diff --git a/contrib/packs/tests/test_action_download.py b/contrib/packs/tests/test_action_download.py
--- a/contrib/packs/tests/test_action_download.py
+++ b/contrib/packs/tests/test_action_download.py
@@ -81,6 +81,22 @@ def mock_is_dir_func(path):
     return original_is_dir_func(path)
 
 
+def mock_get_gitref(repo, ref):
+    """
+    Mock get_gitref function which return mocked object if ref passed is
+    PACK_INDEX['test']['version']
+    """
+    if PACK_INDEX['test']['version'] in ref:
+        if ref[0] == 'v':
+            return mock.MagicMock(hexsha=PACK_INDEX['test']['version'])
+        else:
+            return None
+    elif ref:
+        return mock.MagicMock(hexsha="abcDef")
+    else:
+        return None
+
+
 @mock.patch.object(pack_service, 'fetch_pack_index', mock.MagicMock(return_value=(PACK_INDEX, {})))
 class DownloadGitRepoActionTestCase(BaseActionTestCase):
     action_cls = DownloadGitRepoAction
@@ -520,7 +536,7 @@ def side_effect(ref):
 
             # Fool _get_gitref into working when its ref == our ref
             def fake_commit(arg_ref):
-                if arg_ref == ref:
+                if not ref or arg_ref == ref:
                     return gitref
                 else:
                     raise BadName()
@@ -572,3 +588,28 @@ def test_run_pack_download_local_directory(self):
         destination_path = os.path.join(self.repo_base, 'test4')
         self.assertTrue(os.path.exists(destination_path))
         self.assertTrue(os.path.exists(os.path.join(destination_path, 'pack.yaml')))
+
+    @mock.patch('st2common.util.pack_management.get_gitref', mock_get_gitref)
+    def test_run_pack_download_with_tag(self):
+        action = self.get_action_instance()
+        result = action.run(packs=['test'], abs_repo_base=self.repo_base)
+        temp_dir = hashlib.md5(PACK_INDEX['test']['repo_url'].encode()).hexdigest()
+
+        self.assertEqual(result, {'test': 'Success.'})
+        self.clone_from.assert_called_once_with(PACK_INDEX['test']['repo_url'],
+                                                os.path.join(os.path.expanduser('~'), temp_dir))
+        self.assertTrue(os.path.isfile(os.path.join(self.repo_base, 'test/pack.yaml')))
+
+        # Check repo.git.checkout is called three times
+        self.assertEqual(self.repo_instance.git.checkout.call_count, 3)
+
+        # Check repo.git.checkout called with latest tag or branch
+        self.assertEqual(PACK_INDEX['test']['version'],
+                         self.repo_instance.git.checkout.call_args_list[1][0][0])
+
+        # Check repo.git.checkout called with head
+        self.assertEqual(self.repo_instance.head.reference,
+                         self.repo_instance.git.checkout.call_args_list[2][0][0])
+
+        self.repo_instance.git.branch.assert_called_with(
+            '-f', self.repo_instance.head.reference, PACK_INDEX['test']['version'])