Software is hard because it has a weak theoretical foundation. Most of the theory that does exist focuses on the static behavior of the software, analysis of the source listing. There is little theory on its dynamic behavior, how it performs under load. To avoid serious network problems software systems are over-engineered with plenty of bandwidth for two or three times the expected load. Without analysis of the dynamic behavior, application designers have no idea of the resources they will need once their software is operational.
Software has the awful propensity to fail with no warning.
For most systems, this is impossible except with lots of custom design that is itself error-prone. Software prototyping has proven its mettle in helping designers avoid these problems in their production systems. Much has been written about the best way to develop software applications. Both prototyping and requirements are necessary. The tried and true process of synthesis and analysis is used to solve Software Engineering problems.