Repository: go
Issue: crypto/subtle: change ConstantTimeLessOrEq to have less undefined behaviour
Labels: NeedsInvestigation

<!--
Please answer these questions before submitting your issue. Thanks!
For questions please use one of our forums: https://github.com/golang/go/wiki/Questions
-->

### What version of Go are you using (`go version`)?

<pre>
$ go version
go1.11.6
</pre>

### Does this issue reproduce with the latest release?

Yes. (With apologies for my vintage go compiler.)

### What operating system and processor architecture are you using (`go env`)?

<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/isis/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/isis/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go-1.11"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.11/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build154905218=/tmp/go-build -gno-record-gcc-switches"
</pre></details>

### What did you do?

<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
-->

I have a better algorithm for the `ConstantTimeLessOrEq` function which works for all inputs in [0, 2^32).  Currently the `ConstantTimeLessOrEq` function only works (as documented) for inputs in [0, 2^31) and is UB for everything else (including negative signed integers, which it takes signed ints). Implementing the algorithm and testing it shows that it works for correctly for positive inputs, which means we could also change the interface to be `ConstantTimeLessOrEq(x, y uint32) int` which would remove all UB, but this is obviously a decision for the maintainers.

The algorithm is:

```go
// ConstantTimeLessOrEq returns 1 if x <= y and 0 otherwise.                                                                                                                                                                                   
// Its behavior is undefined if x or y are negative.                                                                                                                                                                                           
func ConstantTimeLessOrEq(x, y int) int {                                                                                                                                                                                                      
    x32 := uint32(x)                                                                                                                                                                                                                           
    y32 := uint32(y)                                                                                                                                                                                                                           
                                                                                                                                                                                                                                               
    gtb := x32 & ^y32                                                                                                                                                                                                                          
    ltb := ^x32 & y32                                                                                                                                                                                                                          
                                                                                                                                                                                                                                               
    ltb = ltb | (ltb >> 1)                                                                                                                                                                                                                     
    ltb = ltb | (ltb >> 2)                                                                                                                                                                                                                     
    ltb = ltb | (ltb >> 4)                                                                                                                                                                                                                     
    ltb = ltb | (ltb >> 16)                                                                                                                                                                                                                    
                                                                                                                                                                                                                                               
    bit := gtb & ^ltb                                                                                                                                                                                                                          
                                                                                                                                                                                                                                               
    bit = bit | (bit >> 1)                                                                                                                                                                                                                     
    bit = bit | (bit >> 2)                                                                                                                                                                                                                     
    bit = bit | (bit >> 4)                                                                                                                                                                                                                     
    bit = bit | (bit >> 16)                                                                                                                                                                                                                    
                                                                                                                                                                                                                                               
    return int(^bit & 1)                                                                                                                                                                                                                       
}
```

### What did you expect to see?

N/A.

### What did you see instead?

**EDIT:** This is obvious but an example test case which (expectedly) fails the previous algorithm is:

```go                                                                                                                                                                                                                   
    {1, 4294967295, 1},
```

**EDIT #2:** Perhaps less obvious, this algorithm also implements `ConstantTimeGreater` by changing the final line to `return int(bit & 1)`, which is also useful for some of the algorithms in a few post-quantum cryptographic algorithms, particularly [those which require a constant-time sorting network](https://lists.torproject.org/pipermail/tor-dev/2016-May/010908.html).