Explicitly build the inverse matrix

Some drivers does not support inverse function although they advertise
glsl 140 support, and it's faster  to do it manually as we know the
structure of the matrix.