opened
cfromknecht
username_0
This PR optimizes the `wire` packages serialization of small buffers by minimizing the number of borrow/return round trips during message serialization. Currently the `wire` package uses a `binaryFreeList` from which 8-byte buffers are borrowed and returned for the purpose of serializing small integers and varints.

## Problem
To understand the problem, consider calling [`WriteVarInt`](https://github.com/btcsuite/btcd/blob/master/wire/common.go#L538) on a number greater than `0xfc` (which requires writing the discriminant and a 2, 4, or 8 byte value following).

For instance, writing 20,000 will invoke [`PutUint8`](https://github.com/btcsuite/btcd/blob/master/wire/common.go#L538) and then [`PutUint16`](https://github.com/btcsuite/btcd/blob/master/wire/common.go#L548). Expanding this out to examine the message passing, we see:
```
buffer <- freelist

PutUInt8(buffer, discriminant)
w.Write(buffer)

freelist <- buffer

buffer <- freelist

PutUint16(buffer, value)
w.Write(buffer)

freelist <- buffer
```

Each `<-` requires a channel `select`, which more-or-less bears the performance implication of a mutex. This cost, in addition to need to wake up other goroutines and switch executions, imparts a significant performance penalty. In the context of block serialization, several hundred thousand of these operations may be performed.

## Solution

In our example above, we can improve this by only using two `<-`, one to borrow and one to return, as so:

```
buffer <- freelist

PutUInt8(buffer, discriminant)
w.Write(buffer)

PutUint16(buffer, value)
w.Write(buffer)

freelist <- buffer
```

As expected, cutting the number channels sends in half cuts also cuts the latency in half, which can be seen in the benchmarks below for larger `VarInt`s, 

The remainder of this PR is to propagate this pattern all the way up to the top level of messages in the wire package, such that deserializing a message only incurs one borrow and one return. Any subroutines are made to conditionally borrow from the `binarySerializer` if the invoker has not provided them with a buffer, and conditionally return if they indeed were required to borrow.

A good example of how these channel sends/receives can add up is in MsgTx serialization, which is now upwards of 80% faster as a result of these optimizations:
```
benchmark                         old ns/op     new ns/op     delta
BenchmarkSerializeTx-8            683           142           -79.21%
BenchmarkSerializeTxSmall-8       724           143           -80.25%
BenchmarkSerializeTxLarge-8       1476002       182111        -87.66%
```

## Preliminary Benchmarks
```
benchmark                         old ns/op     new ns/op     delta
BenchmarkWriteVarInt1-8           71.9          74.6          +3.76%
BenchmarkWriteVarInt3-8           148           70.9          -52.09%
BenchmarkWriteVarInt5-8           149           71.0          -52.35%
BenchmarkWriteVarInt9-8           147           72.8          -50.48%
BenchmarkReadVarInt1-8            78.8          77.9          -1.14%
BenchmarkReadVarInt3-8            159           87.7          -44.84%
BenchmarkReadVarInt5-8            155           88.3          -43.03%
BenchmarkReadVarInt9-8            158           86.9          -45.00%
BenchmarkReadVarStr4-8            120           119           -0.83%
BenchmarkReadVarStr10-8           138           130           -5.80%
BenchmarkWriteVarStr4-8           101           105           +3.96%
BenchmarkWriteVarStr10-8          103           105           +1.94%
BenchmarkReadOutPoint-8           91.3          28.7          -68.57%
BenchmarkWriteOutPoint-8          78.9          10.2          -87.07%
BenchmarkReadTxOut-8              245           118           -51.84%
BenchmarkWriteTxOut-8             151           93.5          -38.08%
BenchmarkReadTxIn-8               338           139           -58.88%
BenchmarkWriteTxIn-8              238           31.0          -86.97%
BenchmarkDeserializeTxSmall-8     1119          586           -47.63%
BenchmarkDeserializeTxLarge-8     2476063       1275815       -48.47%
BenchmarkSerializeTx-8            683           142           -79.21%
BenchmarkSerializeTxSmall-8       724           143           -80.25%
BenchmarkSerializeTxLarge-8       1476002       182111        -87.66%
BenchmarkReadBlockHeader-8        406           69.1          -82.98%
BenchmarkWriteBlockHeader-8       431           21.1          -95.10%
BenchmarkDecodeGetHeaders-8       13537         12238         -9.60%
BenchmarkDecodeHeaders-8          1025275       236709        -76.91%
BenchmarkDecodeGetBlocks-8        13206         11684         -11.53%
BenchmarkDecodeAddr-8             337977        157519        -53.39%
BenchmarkDecodeInv-8              5990935       1898169       -68.32%
BenchmarkDecodeNotFound-8         6285831       1864701       -70.33%
BenchmarkDecodeMerkleBlock-8      4357          2606          -40.19%
BenchmarkTxHash-8                 1928          1222          -36.62%
BenchmarkDoubleHashB-8            993           969           -2.42%
BenchmarkDoubleHashH-8            932           1029          +10.41%

benchmark                         old allocs     new allocs     delta
BenchmarkWriteBlockHeader-8       4              0              -100.00%
// all others remain 0 alloc

benchmark                         old bytes     new bytes     delta
BenchmarkWriteBlockHeader-8       16            0             -100.00%
// all others remain unchanged
```

## Notes

I'm still in the process of going through and adding benchmarks to top-level messages in order to guage the overall performance benefit, expect more to be added at a later point.

There are a few remaining messages which have not yet been optimized, e.g. `MsgAlert`, `MsgVesion`, etc. I plan to add those as well but decided to start with the ones that were more performance critical.
wire: only borrow/return binaryFreeList buffers at the message level
issue
created
jakesyl
username_1
@jcvernaleo (as per #1530)

- low priority
- enhancement

By the way @username_0 this is awesome!
comment
created
cfromknecht
username_0
FWIW I rebased this PR (locally) over #1684 and reran the `CalcSighHash` benchmarks.
Since calculating the legacy sighash involves [serializing a copy of the original transaction](https://github.com/btcsuite/btcd/blob/master/txscript/script.go#L658),
optimizing our wire encoding speeds up the sighash calculation speed significantly. 

This is a comparison of CalcSigHash before and after applying the wire optimizations
to this branch: 
```
benchmark                  old ns/op     new ns/op     delta
BenchmarkCalcSigHash-8     3619477       1758789       -51.41%

benchmark                  old allocs     new allocs     delta
BenchmarkCalcSigHash-8     801            712            -11.11%

benchmark                  old bytes     new bytes     delta
BenchmarkCalcSigHash-8     1293354       1290504       -0.22%
```

Using an 80% transaction serialization speedup to ballpark, this indicates roughly 2/3 of
the latency incurred by`CalcSigHash` is transaction serialization, and with this change it
becomes 1/4.
comment
created
onyb
username_2
@username_0 Could you please rebase this PR, since https://github.com/btcsuite/btcd/pull/1769 landed recently?
comment