Alright, with some benchmarking, I think we can take out the `@inbounds` call. The overhead of the checkbounds is pretty small 10-15% as far as I can tell and that's ok with me.