Clean up how framelib matrices are constructed