Update matrix.c

Fixed bug causing stack overflow with large matrices (changed to dynamic allocation in matrixDet)