Import Data.Bytestring unconditionally in Crypto.Number.Serialize.

This fixes a compilation error when integer-gmp >= 0.5.1 is in use:

```
Crypto/Number/Serialize.hs:62:11: Not in scope: ‘B.singleton’
```