To add tho this, you could probably rewrite chunks of this function with generators to make it more efficient.