The Mainframe Automated Testing Need
By: Sujay Solomon
Tags: Test Automation, Automated Testing, Quality Assurance, Open Source, Mainframe

The Mainframe Automated Testing Need

Software production is still a relatively young industry in the grand scheme of things. Think about it — we’ve been mass-producing automobiles for over a century. Given worldwide demand for automobiles, the industry had no choice but to rapidly streamline and accelerate production. Ford installed the very first assembly line in 1913 and it didn’t take long before lean production methodologies found their roots in Toyota’s 1930 operating model.

Photo by Birmingham Museums Trust on Unsplash

Today, the U.S. Department of Transportation reports the average life of a vehicle to be 12 years, engineered to last around 200,000 miles. Wind back time to the 1960s and the expectation was about half that — 100,000 miles. This increase in quality and longevity did not happen on its own. Its continuous improvement to production methodologies, including quality automation, that ultimately results in a better product over time.

Licensed under the Creative Commons license by TGW Logistics Group

It’s hard to imagine that the phrase “software is eating the world” was coined a decade ago, but it continues to ring true. Can you imagine going an hour without interacting with something driven by software today? Even your lightbulbs are “smart”. The demand for software couldn’t be higher. With such a sustained high demand, we’ve had to innovate our software production methodologies to keep up.

Along came DevOps with the concept of an automated pipeline for software development where quality is embedded during the production process and not an obstacle to cross at the end. Quality automation is no longer a choice, it’s the norm. In the last decade, this philosophy has permeated through software development culture, resulting in software products that are produced at rapid speeds, but also engineered to last.

We’ve made great strides in terms of efficiency and quality in the production of new software. But, what about all the software that was built before this? Mainframes continue to be the backbone of the finance industry that sits at the heart of our entire economy. Like a home that lasts the test of time, these applications continue to be rock solid and excel at what they were designed to do. But as homes benefit from renovations from time to time, these applications stand to gain from upgrades.

But what are these upgrades? If I’m having higher utility bills, I might choose to improve the insulation in my home. If I want greater security, I might install a security system.

A common need of mainframe applications is to ensure quality across the changes being made and a desire to make changes more frequently without fear of quality degradation. Modern applications can be updated and deployed quickly due to automated testing that was built ground up. This is foundational to DevOps best practices. Every layer of the application typically has automated testing — unit testing, integration testing, system testing. All these tests have their purpose, but if you ask an SRE (Site Reliability Engineer) which test results they’re most invested in before a change makes it to production, it would be system testing, followed by integration testing consisting of functional and non-functional tests.

Mainframe applications usually do not have their tests automated. The right tools were not always available for automation and inertia had set in, limiting any serious efforts to automate testing retroactively. There’s also the small matter of automation requiring coding skills that traditional mainframe quality testing staff didn’t possess.

Before we try to solve the need for automated testing, let’s take a look at how testing is done for mainframe applications today.