shouldInlinePrimOp: Fix Int overflow

There were two overflow issues in shouldInlinePrimOp. The first one is
due to a negative CmmInt literal being created if the array size was
given as larger than 2^63-1 (on a 64-bit platform.) This meant that
large array sizes could compare as being smaller than
maxInlineAllocSize.

The second issue is that we casted the Integer to an Int in the
comparison, which again meant that large array sizes could compare as
being smaller than maxInlineAllocSize.

The attempt to allocate a large array inline then caused a segfault.

Fixes #9416.
Add some Haddocks to SMRep
StgCmmPrim: add note to stop using fixed size signed types for sizes

We use fixed size signed types to e.g. represent array sizes. This
means that the size can overflow.