I think this was definitely a software failure. The people of Miramichi did not order or design the system, they were just users. Miramichi is a call center, a help desk essentially. In a new software system, the rules to run the relevant part of the organization should be well laid out and easy to use. With Oracle PeopleSoft, even when people knew there was a problem, no one was able to fix it because the software didn't allow it. People didn't get paid, some federal employees apparently lost their houses because they weren't getting payed. I personally know people who told me they were getting paid incorrect amounts for years an no one could fix it. The Government of Canada should never have tried to shoehorn an off the shelf Oracle system to run payroll for a whole country's employees. When you are that big and that complex of an organization, the system needs to be created and tailored to your need. This is almost an information theoretic truth. There is an exponential number of corner cases in such a large organization.

When people think of buying business software systems, they often imagine it's like buying a car or a an app on their phone. A better metaphor for software infrastructure is the renovation or replacement of floors in a skyscraper.

To function properly, business software usually needs to interact or interface with data, software and hardware infrastructure that is in countless existing systems.

The complexity in today's IT infrastructure is mind boggling. It is the result of decades of evolution. The data in existing systems is often held in tens of thousands of individually defined formats and schemas, it flows through thousands of specialized channels that have evolved over years with mechanism for propagation of errors, for dealing with exceptions, for ensuring constraints are in place for security, while allowing special cases when needed. There are rules to ensure redundancy, high availability, backups, to support different user interfaces, different types of computers and mobile devices. There are compatibility layers for different systems, hardware, operating systems, databases. In the case of Canadian government payroll, in the old code there were embedded rules about thousands of different positions negotiated over decades by a bunch of different unions. The business rules in the old code took into account, different programs created in different areas by different governments in different regions.

Skyscrapers can also seem simple from the outside, often being made of flat surfaces forming plain rectangular shapes. But a lot more lies under their skin. There are miles of electrical wires, of plumbing, heating and ventilation, security systems that have to connect and work together. There are stairwells, elevator shafts that have to be planned and positioned for safety.

When parts of skyscrapers get old and outdated, you never see anyone suggesting buying a pre-made piece of skyscraper and shoving it in place of the floors that are outdated. Yet this is what people try to do with software infrastructure.

It's as if an organization having a building with floor 20 to 30 needing updating decided to buy a 10 floor building to put in it's place. They sign the contract and then start working on the details. Putting one building inside another is not easy. You will have to renovate the five floors bellow the new floors and also above, so that the wiring, plumbing stairwell and elevator can connect. The elevators from the two buildings is a particularly difficult problem. They won't connect. As a workaround you might get people to take a detour and a stairwell for one floor to the old elevator, ah and add a two floor high elevator for handicap access next to the stairwell. After a few runs of sewage pipes on the outside wall of the building and running electrical wires in the air crossing from the building next door, your floor twenty to thirty now satisfy the contract and the company that sold you the ten floors considers the job done. Of course it works much worst than the old 10 floors and of course you also now have 20 barely functional floors instead of 10 outdated one. But what are you going to do? You already paid way over budget to hack this mess together. It is now very unstable, if you change anything the whole building might collapse and the company you contracted has pulled a trick. The blueprints to the building (aka. the source code) is proprietary so no other company can work on it.

Undoing the damage would mean completely rebuilding 20 floors (and the building needs to keep operating while this is happening and there is only one company that has access to the blueprints so they can charge anything they want and do a dismal job at it and leave you in a position where you have to perpetually give them more money to keep things minimally functional.

When you go to them because nothing is working, they will steer you towards replacing more of your floors with more of their buildings and break more floor above and bellow while taking away your access to the blueprints and securing their monopolistic position.

Successful businesses never allow themselves to be put in such a bad position for very long. Just recently Amazon announced that they got rid of all their Oracle databases. Governments should start that work as soon as possible. Yes it may require expensive rebuilding of parts of the infrastructure piece by piece, but in the long run it will be much less expensive.

Always renovate room by room, floor by floor while making sure the blueprints remain available so you can use different contractors when one turns out to be a lemon.