Network now converges with more layers than 3, problem was non-symmetrical (in relation to 0) random weights when initialized