Repository: mockito/mockito
Issue: Make the build script aware of the release branch
Labels: continuous integration

Releasing from the `master` branch has advantages developer wise, however when users that are hit by a bug and that `master` has significantly changed we cannot propose a stable enough release (example 2.0 betas releases, even the 1.10.x series had some instabilities API wise).

In order to enable the team and contributors in general to fix issues while still allowing bigger work on the next version I propose the following change in branch and release _process_.
1. Branch out to `release/2.x`
   1. Release `2.0.0`
   2. Patch issue
   3. Release `2.0.1`
   4. Additional work on 2.1
   5. Release `2.1.0`
   6. ...
   7. Forget about `2.1.x`
   8. Critical fixes only get released
2. Upgrade `master` to `3.0.0-beta-<build number>`
   1. Start mockito `3.0.0` progress there (like java 8 only, drop APIs)

Also with recent change in the build script we could now release and check the release version with the branch.

See 
- https://github.com/mockito/mockito/commit/b6a402f16e9222605638efe9b57eccc80a197f9e#diff-33cf41d9669aaef72ca19c641dc57d47R68
- https://github.com/mockito/mockito/issues/586#issuecomment-242076979
- https://github.com/mockito/mockito/pull/483#issuecomment-231323266


Issue comments:
@bric3, this is great. Thank you for finally describing the workflow!!!

~~I'll try to work on this this week. Yet this ticket is not mandatory for releasing (hence no milestone assignment).~~

EDIT in the end I couldn't find time this week fo this

The plan makes perfect sense. Let's zoom into implementation details. For example:
1. Create 2 active branches: "master" and "release"
2. Set version.properties in "master" to 3.0.0-beta.1, automatically publish per push
3. Set version.properties in "release" to 2.1.0-rc.1, automatically publish per push (?)
   a. At some point change version.properties in "release" to "2.1.0", automatically publish per push

I know that travis can now get version from commit message. How would this work for keeping 2 active branches / publishing RCs?

If you want to publish a RC for the v2 branch, you checkout that branch, then commit with `Prepare release candidate [ci-release v2.1.0-RC.1]` on that branch and it should be good to go.

I'd rather go for a branch out model like described in https://github.com/mockito/mockito/issues/594#issue-172969427, so we (and interested parties) can still release a important patch for an old mockito version.

@szczepiq I'm waiting on the release note of #582 to branch out to `release/2.x`

That should be working fine right? With the versionproperties separate in each branch (and the manual releases).

Yes, but not with a single `release` branch.

> 1. Create 2 active branches: "master" and "release"

Oh sorry, I misread that. Yes we must have a release branch for every major semver version. Reading is hard sometimes :stuck_out_tongue_winking_eye: So +1 for me for @bric3 model, which I also was trying to describe.

When do we use magic incantation in the commit message and when not?

There is an approach with fully automated releases:
1. Create "v2" branch (or something like that), that still publishes Beta.
2. Test it out, so that Beta is neatly released from "v2".
3. Change version in "v2" into RC, remove top of the release notes, push.
4. Iterate on the fixes, every push creates new RC.
5. At some point, change version to "2.1.0", push.
6. Any push to "v2" publishes new 2.1.*

For 2 active dev branches (e.g. mockito3 beta in master and v2) we need to split release notes (e.g. notes/v2.md and notes/v3.md). It's not blocking v2 release, we can do it later.

Awesome, I'm getting excited, we'll have RC before JavaOne! :)

> Any push to "v2" publishes new 2.1.*

I would instead have the drone only push betas. E.g. 2.1.1-beta.*.

On Thu, 1 Sep 2016, 07:18 Szczepan Faber, notifications@github.com wrote:

> When do we use magic incantation in the commit message and when not?
> 
> There is an approach with fully automated releases:
> 1. Create "v2" branch (or something like that), that still publishes
>    Beta.
> 2. Test it out, so that Beta is neatly released from "v2".
> 3. Change version in "v2" into RC, remove top of the release notes,
>    push.
> 4. Iterate on the fixes, every push creates new RC.
> 5. At some point, change version to "2.1.0", push.
> 6. Any push to "v2" publishes new 2.1.*
> 
> For 2 active dev branches (e.g. mockito3 beta in master and v2) we need to
> split release notes (e.g. notes/v2.md and notes/v3.md). It's not blocking
> v2 release, we can do it later.
> 
> Awesome, I'm getting excited, we'll have RC before JavaOne! :)
> 
> —
> You are receiving this because you commented.
> 
> Reply to this email directly, view it on GitHub
> https://github.com/mockito/mockito/issues/594#issuecomment-243978550,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/AFrDbxJRdQ7Q3PHv8p0kxCZveqCVpp9Fks5qlmA6gaJpZM4JsE40
> .

No more betas please :) Continuous delivery is signature feature of Mockito project.

While that is true, for every pull request we would release a new patch version. I am not a fan of that. Preferably update less, but more meaningful for the end-user. If you are continuously producing new non-beta artifacts, I think end-users tend to wait with upgrading instead. It's sad that gradle/maven can't import from master instead, but I think we should try to keep our semver versions clean.

This is an old debate, that we had on the mailing list a couple of years ago.

@szczepiq I know you are an advocate of each commit can be a release, I heard about your talk. This is really nice pushing forward for continuous delivery however I don't agree that every single commit should be released especially on libraries. For some projects it maybe ok, but for some I'd be more conservative.

For example 1.10.x series was nice but felt less stable to many users. Some merged PR introduced the Mockito rule and the API had to be changed. Some incorrect behavior had to be fixed, some were fixed later in the beta. I'm not sure this builds trust in the library.

Also with this big changes cannot fit in your model : should we have released in between PR of the JUnit feature, after all there was enough interesting and working features already. 
For mockito 3 we will target java 8, there's a lot of clean up, maybe some rewriting, doing so we may break things, that happens even with tests, this phase should not be seen as something else than a beta.

> For 2 active dev branches (e.g. mockito3 beta in master and v2) we need to split release notes (e.g. notes/v2.md and notes/v3.md). It's not blocking v2 release, we can do it later. 

Exactly. Note that 2.x branch should see most activities, but the scope is limited. And after 2.1 we could declare 2.x in_maintainance_ mode.

I'm happy to rekindle the discussion. If you convince me, I win win because I learn more :)

> Also with this big changes cannot fit in your model

Why cannot the big feature be developed in a branch?

> For example 1.10.x series was nice but felt less stable to many users.

"felt less stable" - do you have concrete data?

Some thoughts:
- users don't have to upgrade to every version
- we could do minor version bump every month or so, combine the release notes and do bigger announcement

> Some merged PR introduced the Mockito rule and the API had to be changed
> Some incorrect behavior had to be fixed, some were fixed later in the beta.

I'm not convinced that this is a problem of CD. This could happen in the traditional release model, too.

I'm a fan of CD because:
- it forces to care about quality, documentation, compatibility of every PR
- it gives features to users faster
- it provides features incrementally, the smaller the change, the lower chance of regression
- it helps with project maintenance - users test and pick up new versions faster

Let's say that we don't do CD. How would the release look like? You would release once in a while using magic String in commit message?

@szczepiq I don't say abandon CD, I like the CD, for the reason you mentioned. But let's not make a  release per se, like today maybe label them as `beta` or `build` this builds shows progress and still allows interested users to try mockito current work.

> > Also with this big changes cannot fit in your model
> 
> Why cannot the big feature be developed in a branch?

Yes they can, but sometime those are incomplete or buggy. Like having thread safety, or whatever else. 
And even so when these features land it is just possible to go back on some features, APIs, behaviour that was developed in a previous release.

> > For example 1.10.x series was nice but felt less stable to many users.
> 
> "felt less stable" - do you have concrete data?

Actually this _data_ comes the issue and PR we had at that time, questions asked on SO and from my own colleagues experience. However I don't have those question/gh issues in mind as it's been a long time ago.

> - users don't have to upgrade to every version

Indeed.

> - we could do minor version bump every month or so, combine the release notes and do bigger announcement

Yeah that works, but with previous point I think this could just be intermediary _preview_ / _milestone_ releases along with other regular beta builds.

> Yes they can, but sometime those are incomplete or buggy. Like having thread safety, or whatever else.
> And even so when these features land it is just possible to go back on some features, APIs, behaviour that was developed in a previous release.

Let's not push buggy or incomplete features :) The stability problem should be solved by high quality PRs, well designed, documented, and tested.

We can push "build" versions, etc. from branch if we see that it's a good idea.

> Actually this data comes the issue and PR we had at that time, questions asked on SO and from my own colleagues experience. However I don't have those question/gh issues in mind as it's been a long time ago.

It seems there aren't concrete datapoints at this time. I'd rather not change the CD strategy, wait for concrete data, re-evaluate as needed.

In the end I couldn't work on it this week, I'll try next week.


Pull request review comments:

Patch that resolved the issue:
diff --git a/doc/release-notes/official.md b/doc/release-notes/official.md
index ce90a6ede3..5058739575 100644
--- a/doc/release-notes/official.md
+++ b/doc/release-notes/official.md
@@ -1,3 +1,26 @@
+### 2.1.0-beta.125 (2016-09-08 03:42 UTC)
+
+* Authors: 4
+* Commits: 24
+  * 15: Szczepan Faber
+  * 5: Continuous Delivery Drone
+  * 3: Brice Dutheil
+  * 1: Divyansh Gupta
+* Improvements: 3
+  * Documentation: 1
+    * Update documentation links in travis config comments [(#558)](https://github.com/mockito/mockito/pull/558)
+  * Remaining changes: 2
+    * Ensured javadocs are correct [(#626)](https://github.com/mockito/mockito/pull/626)
+    * Tweak javadoc [(#625)](https://github.com/mockito/mockito/issues/625)
+
+### 2.1.0-beta.124 (2016-09-07 04:36 UTC)
+
+* Authors: 2
+* Commits: 12
+  * 11: Szczepan Faber
+  * 1: Continuous Delivery Drone
+* No notable improvements. See the commits for detailed changes.
+
 ### 2.1.0-beta.123 (2016-09-06 15:01 UTC)
 
 * Authors: 4
diff --git a/gradle/release.gradle b/gradle/release.gradle
index 78ed82995b..d8005a59a8 100644
--- a/gradle/release.gradle
+++ b/gradle/release.gradle
@@ -83,8 +83,8 @@ task("releaseNeeded") {
         def skippedByCommitMessage = commitMessage?.contains(skipReleaseCommitMessage)
         def customVersion = (commitMessage =~ customReleaseVersionPattern)
         ext.needed = dryRun || customVersion.matches() || (pr == 'false' && isReleasableBranch(branch) && !comparePublications.publicationsEqual && skipEnvVariable != 'true' && !skippedByCommitMessage)
-        logger.lifecycle("Release needed: {}, branch: {}, pull request: {}, dry run: {}, publications equal: {}, skip env variable: {}, skipped by message: {}, customVersion: {}.",
-                needed, branch, pr, dryRun, comparePublications.publicationsEqual, skipEnvVariable, skippedByCommitMessage, customVersion.matches())
+        logger.lifecycle("Release needed: {}, releasable branch: {}, pull request: {}, dry run: {}, publications equal: {}, skip env variable: {}, skipped by message: {}, customVersion: {}.",
+                needed, isReleasableBranch(branch), pr, dryRun, comparePublications.publicationsEqual, skipEnvVariable, skippedByCommitMessage, customVersion.matches())
     }
 }
 
@@ -94,7 +94,7 @@ bintrayUpload {
     doFirst {
         logger.lifecycle troubleshootingInfo
         if (dryRun) {
-            logger.lifecycle "Dry-running the release"
+            logger.lifecycle "Dry-running the release! Although 'bintrayUpload' is executed, it won't upload any binaries ('bintrayUpload.dryRun' property is set)."
         }
     }
 }
@@ -118,7 +118,8 @@ releaseSteps {
 
     step("ensure good chunk of recent commits is pulled for release notes automation") {
         //Travis default clone is pretty shallow
-        run "git", "pull", "--depth", "500"
+        //Loading extra 1K commits (1.x -> 2.x had ~700 commits)
+        run "git", "pull", "--depth", "1000"
     }
 
     def gitAuthor
@@ -128,7 +129,8 @@ releaseSteps {
     step("commit updated javadoc into gh-pages branch") { commitUpdatedJavadoc(buildInfo) }
 
     /*
-            Now we'll start operating on master. This introduces a problem - someone might have pushed changes *after* release process has started
+            Now we'll start operating on branch that we want to release.
+            This introduces a problem - someone might have pushed changes *after* release process has started
             What can happen:
              - bintrayUpload will fail saying that the version is already released
              - git push fails saying that the pull is needed
@@ -136,7 +138,7 @@ releaseSteps {
              - see 'TROUBLESHOOTING' section at the top of this file
     */
 
-    step("start operating on master") { run "git", "checkout", "master" }
+    step("start operating on $System.env.TRAVIS_BRANCH") { run "git", "checkout",  System.env.TRAVIS_BRANCH }
 
     step("update release notes") { project.notes.updateReleaseNotes() }
 
@@ -146,11 +148,11 @@ releaseSteps {
     step("create new version tag") { createTag(buildInfo, "v${currentVersion}".toString()) }
             .rollback { run "git", "tag", "-d", "v${currentVersion}".toString()}
 
-    step("commit incremented version on master") { commitIncrementedVersion(currentVersion, buildInfo, project.versionFile) }
+    step("commit incremented version on $System.env.TRAVIS_BRANCH") { commitIncrementedVersion(currentVersion, buildInfo, project.versionFile) }
             .rollback { run "git", "reset", "--hard", "HEAD^" }
 
     step("push changes to all involved branches") {
-        def pushCommandLine = ["git", "push", pushTarget, "master", "gh-pages", "v$currentVersion", "-q"]
+        def pushCommandLine = ["git", "push", pushTarget, System.env.TRAVIS_BRANCH, "gh-pages", "v$currentVersion", "-q"]
         if (dryRun) {
             pushCommandLine << '--dry-run'
         }
diff --git a/src/main/java/org/mockito/Mockito.java b/src/main/java/org/mockito/Mockito.java
index 35c06f5496..7baeea8cbe 100644
--- a/src/main/java/org/mockito/Mockito.java
+++ b/src/main/java/org/mockito/Mockito.java
@@ -73,14 +73,14 @@
  *
  * <h3 id="0">0. <a class="meaningful_link" href="#verification">Migrating to 2.1.0</a></h3>
  *
- * In order to continue improving Mockito and further improve the unit testing experience, we want you to upgrade to 2.0.
+ * In order to continue improving Mockito and further improve the unit testing experience, we want you to upgrade to 2.1.0!
  * Mockito follows <a href="http://semver.org/">semantic versioning</a>
  * and contains breaking changes only on major version upgrades.
  * In the lifecycle of a library, breaking changes are necessary
  * to roll out a set of brand new features that alter the existing behavior or even change the API.
  * We hope that you enjoy Mockito 2.1.0!
  * <p>
- * List of breaking changes:
+ * List of breaking changes: TODO 596
  * <ul>
  *     <li>Mockito is decoupled from Hamcrest and custom matchers API has changed, see {@link ArgumentMatcher}
  *     for rationale and migration guide.</li>
@@ -1233,7 +1233,7 @@ public class Mockito extends ArgumentMatchers {
      * <code>ReturnsSmartNulls</code> first tries to return ordinary return values (see {@link ReturnsMoreEmptyValues})
      * then it tries to return SmartNull. If the return type is final then plain null is returned.
      * <p>
-     * <code>ReturnsSmartNulls</code> will be probably the default return values strategy in Mockito 2.0.
+     * <code>ReturnsSmartNulls</code> will be probably the default return values strategy in Mockito 3.0.0
      * <p>
      * Example:
      * <pre class="code"><code class="java">
diff --git a/version.properties b/version.properties
index b0469d5032..0035f85bad 100644
--- a/version.properties
+++ b/version.properties
@@ -1,2 +1,2 @@
-version=2.1.0-beta.124
+version=2.1.0-beta.126
 mockito.testng.version=1.0


Test patch: