Simplify the SSE stuff by removing some types and object layers.

Intent was a simpler/friendlier API, but ended up reducing allocs as well.