“Software testing is loaded with unchallenged ideas from the last 20 or so years, and changing its perception works against ingrained bias and prejudice.”
I recently came across this line in a blog post by Keith Klain and, as someone who loves to challenge ingrained biases and prejudices anywhere I can, it definitely caught my attention. The blog goes on to illustrate some interesting new ways of testing software, highlighting one company in particular that’s really getting it right.
But, by the end of the blog post, I was only more curious about that one line and the journalist in me started getting antsy… so I arranged an interview with the author.
Before I share the entire conversation, I wanted to address the original point that pulled me in… Just in case it piqued your curiosity, too.
According to Klain, the whole industry is underpinned by a false analogy that’s made between software testing and manufacturing. It may be false, but it’s not hard to see where it came from. Software is a product people buy, so it makes sense that it’d be treated like every other thing on the assembly line. To be fair, it may have worked perfectly fine when software was limited to being embedded in devices or shipped on CDs. But, like everything else, the Internet and wireless revolution pulled the strategy rug out from under software testing. So, if you’re actually testing Web based software today, that analogy—and the bias and protocol that result from it—is probably a major source of frustration for you.
Before we get into the details about why the bias towards seeing software testing as a manufacturing process doesn’t fly and what we can do about it, I’d like to step back a bit and acknowledge that this isn’t a new discussion. I found this blog from a few years ago calling for “manufacturing no more,” with the astute suggestion of replacing the manufacturing analogy with an attending physician for web applications:
“We just don't write or release software the way we used to. Software isn't so much built as it is grown. Software isn't shipped ... it's simply made available by, often literally, the flip of a switch. This is not your father's software. 21st century development is a seamless path from innovation to release where every phase of development, including release, is happening all the time. Users are on the inside of the firewall in that respect and feedback is constant. If a product isn't compelling we find out much earlier and it dies in the data center. I fancy these dead products serve to enrich the data center, a digital circle of life where new products are built on the bones of the ones that didn't make it.
In our father's software and Deming's model we talk about quality control and quality assurance while we play the role of inspector. In contrast, my job seems much more like that of an attending physician.”
I really like the physician analogy, even though it may be a bit of a stretch. For now, it’s a healthy one. We need to see test cases more as living organisms that must respond to changes and adapt to input. We need to keep records and track data in relationship to a dynamic system that lives in a habitat so to speak, not an unchanging object. With that in mind, let’s dive into the issue.
Many people who are involved in the software industry but don’t test software think you should (very easily) be able to break a test system down into predictive components. Often they are the ones looking to either make or save money by counting and quantifying all the steps it would take to sufficiently test software. So, it necessarily needs to be predictable—you can’t determine how much you could save or make if you can’t predict it. Nothing wrong with wanting to make and save money, of course. It’s the insistence that predicting test cases makes for adequate test cases that ultimately undermines quality efforts.
This is essentially trying to create a commodity around a test case so that companies can sell testing services by promising to break down a system into hundreds or even thousands of test cases and charge for each one. Or, for a QA manager, this strategy might be understandably motivated by the need to plan a budget. But that approach robs a tester of much needed flexibility, the freedom to adapt to previous test results and their stakeholders, to change things as they get to know the software, and ultimately to produce intelligent tests that work well at identifying defects.
The other major influence supporting this old paradigm is the trend of shipping software to be tested overseas or off site. The need to control from afar and give directive guidance has deepened the manufacturing correlation rut tremendously. Again, it’s understandable. We humans usually equate control with security. Unfortunately, it’s not true in this case.
A recent Cambridge University study cited that the U.S. economy looses $312 billion per year from software defects. That’s a huge number, but there are about as many ways to calculate those costs as the number itself. It’s like trying to count all the impoverished children in the world—it depends so much on how and where you look and often by who is doing the looking. So, I personally prefer to track individual stories, and particularly appreciate this slide show Bloomberg put out a few weeks ago on the most damaging software defect catastrophes over the past 50 years. The point is, either way you look at it, we have a lot of room for improvement.
We all have biases, be they from language, education, culture, nation, gender, age or anything else that makes us human. In some ways, we can never be completely free from bias, but we can and need to be aware of its influence – especially when it becomes an obvious roadblock to our development. Luckily, the bias towards a manufacturing model in testing has already proven to be something we can change—with a little education, patience and persistence.
Keith Klain and others like him are taking a few different approaches to shifting this tide—one from a leadership or managerial perspective that involves a major overhaul of entire testing teams, and the other through educating the individual tester.
Only a few years ago, Klain took over Barclays’s Global Test Center. While he found that the team was extremely proficient in test management, “their misaligned priorities had the effect of continually hitting the bull’s eye on the wrong target.” Keith put a system in place “to foster testing talent and to drive out fear—abandoning worthless metrics and maturity programs, overhauling the training regime, and investing in a culture that rewards teamwork and innovation.” To hear more on how Klain implemented these radical changes, check out this YouTube video.
Obviously, the kind of overhaul Klain directed at Barclays takes a highly motivated, dedicated and persistent leader—let’s be frank, that’s a rare person. However, he and other influencers in the quality industry are working another angle as well. And this one not only is dedicated to revolutionizing software testing, they are out to put a dent in poverty as well. Through software testing training programs targeting under-served youth and adults in several major U.S. cities, Per Scholas, is doing both. Per Scholas trains testers to be contextual thinkers from the beginning and to bring the “manufacturing no more” values to their future work places. In the process, they are seeding a new trend that has the potential to shift old biases and pull the entire industry out of a pretty deep rut.
Let us hear your thoughts, biases, experience, frustration, etc. about your own work as a tester or test manager in the comment section. And stay tuned for the full video interview with Keith Klain, as well as an exciting new video series SmartBear is producing with him (think leaders in the software testing industry… debating).
We provide technological advances to offer our clients a full range of infrastructure design, security, internet solutions and hosted voice services. Our experts work with you one-on-one to ensure we tailor cost-effective solutions that address your specific requirements. Whether you are moving to a new location, building-out existing space or just looking to upgrade your old voice and data systems, we find you the solutions you need at the price you can afford.
Our business was founded in 1998 and we continue to offer the same friendly, personalized service. As an end-to-end telecommunications consulting and management company, we take pride in consistently providing our clients with an unsurpassable level of expertise and support regarding all their telecommunications needs.
When you call us for help with your telecommunications needs, you get dependable connectivity to keep you in touch with your customers and keep your business running smoothly. Our products are backed by expert consulting, unbelievably fast installations, proactive personal service and a U.S.-based technical support team that works around the clock.
We have grown to become one of the largest Hosted PBX business/SIP providers in the country, and have won numerous awards in our field. At the same time, we have established a reputation as being industry leaders who accurately meet the needs of our clients by providing not only excellent workmanship, but impeccable customer service as well.
We help clients in multiple industries choose the right telecom solutions, including, but not limited to, those in healthcare, legal, banking and financial, retail and real estate. We specialize in telecom voice features, Internet solutions and infrastructure services that help you stay connected and transport your data and messages where (and when!) they need to go.
Let today be the day you make the smart choice when it comes to your company’s telecommunications. Call us at Smart Choice Communications and see how we can improve your company’s connectivity.
Instantly join our latest and greatest trade copier service – Following your signup below you will be guided to the trade copier management page.