Order tags by semver build metadata

I use semver build metadata to track patches on top of third party libraries, e.g. (see https://github.com/trask/flot/tags)

(upstream) library release tags:
* 0.8.0-beta
* 0.8.1

my patch tags:
* 0.8.0-beta+patch.1
* 0.8.1+patch.1

I like this convention because it's clear which version of the library was patched, and it still allows for multiple patches on a given library version using `+patch.1`, `+patch.2`, etc.

However when I add `"trask/flot#0.8.1+patch.1"` to my bower.json, it downloads the tag `0.8.1` from my repo, not the tag `0.8.1+patch.1`:

    bower flot#0.8.1+patch.1    not-cached git://github.com/trask/flot.git#0.8.1+patch.1
    bower flot#0.8.1+patch.1       resolve git://github.com/trask/flot.git#0.8.1+patch.1
    bower flot#0.8.1+patch.1      download https://github.com/trask/flot/archive/0.8.1.tar.gz

I realize ordering by build metadata is not required by the semver spec, which explicitly says "Build metadata SHOULD be ignored when determining version precedence".

And there are simple workarounds to this issue, for example removing tag `0.8.1` from my repo so bower picks up tag `0.8.1+patch.1`, or using a commit hash in my bower.json.

But I definitely would have been least surprised if `"trask/flot#0.8.1+patch.1"` in my bower.json had downloaded tag `0.8.1+patch.1`.

Now that I think about it though, once I have a tag `0.8.1+patch.2`, I may be surprised that `"trask/flot#0.8.1+patch.1"` now downloads tag `0.8.1+patch.2` (at least with my PR as currently written).

Or maybe I'm going about this all wrong?  Is there a best practice for this use case already?  Thanks.