Downward Spiral Syndrome because of Techncial Debts and Time crunch.
By: Girish Kurup
Tags: Digital Transformation, Digital Marketing, Technology, Technical Debt, Procrastination

Downward Spiral Syndrome because of Techncial Debts and Time crunch.

Picture form shutterstock.com

Industry 4.0 is the age of Data and Digitization. Every traditional non IT businesses are investing heavily to digitize their business processes to maximize their customer reach, sales and revenues.

Traditional business (Brick and Motor Incumbents) are trying to follow success stories of Tech giants like Google, Amazon, Netflix etc to produce Softwares at scale. All the business process automation softwares, Marketing Software, Sales software, CRMs, IT automation softwares ,supply chain softwares etc are competing within IT organization for maturity. There is constant hunger to produce features at scale to meet the digital strategy requirements.

Traditional businesses(Incumbents) have Traditional monolithic IT. Changing the traditional IT mindset to adopt mindset for micro service modular IT is challenge . The digital strategy demand

experimentation and hypothesis testing of new features . Failure of experiments is taken positively

Innovation at scale

the adoption of cutting edge technologies to deliver Software at scale.

Some of the startup companies powered by the angel investors are able do more with less and they give tough completion to incumbents in their common business lines. These startup small in size are able to eat away marketshare form the incumbents. Incumbents being huge in size are slow in adopting Digital at Scale.

Success of Incumbents Technology departments lies in the hands of technologists . They are the real heroes ( how ) part of ( what). “What” is the strategy adopted by senior leadership. End of day it’s the technologists that realize the dreams of strategy by engineering, inventing or simplifying the current businesses processes. They chain up various technologies to meaningful automated business services.

Below are the basic contributions of Technologists (architects, developers, Testers, Infrastructure specialists)

identify features and defects ,

design and build features and fix defects

Test the features and defect fixes

Identify Security and compliance risks

Identify Technical debts

Below strategies are commonly adopted by incumbents implementing digital strategy.

Devops, Lean and Agile framework to bring features to production every month or every week or every day or even every hour. Extending Devops to business not just limited to IT.

Automation framework to reduce the human reliance factor in Business processes and IT. Use of RPA to automate business processes. Building bots.

Production deployment strategy A/B testing. Canary incremental deployment strategy

Rehost their current workloads, compute and storage from their premises(local datacenters) to cloud (AWS, AZURE , GCP ) . Replatform their current business applications.

Revamp the current Application architecture. Revamp the current Technology and Infrastructure architecture. Revamp the CRM, Marketing , Sales Applications, supply chain back office Applications, Business Intelligence Applications. Set up Infrastructure landscape to embrace IOT, BIGDATA ans conduct Datascience projects.

For being Digital, many customer centric applications are built to enhance the customer experience with business processes, operations, products or services. These Applications generate huge quantity of dataset. New innovation features are invented to gain insights form these CUSTOMER and SUPPLY CHAIN Dataset.

One or more of the above strategies manifest into functional or infra features. These features find their way into the Agile stories. While building the minimum viable solution, certain technical activities or capabilities are parked for future. Many a times due to pressure and minimal budget, workarounds are tailored into the IT fabric. During Digital journey , IT teams forget about these cutting cornered solutions or workarounds. Most often the bottlenecks are known but teams have no time to fix them. In order to fulfill the business process requirements, technology life cycle features , security risks, are neglected. Many times the feedback from the business , development teams , architects to improve the current status-co is neglected. Habit of unfinished work increases. These workarounds, deferred bottlenecks, deferred technology life cycle features , deferred security &risk features, unfinished works and deferred improvement are termed as Technical debts.

Typical situation:

Stage 1 Nothing going to happen skip the Technical debt. Means no impact of technical debt so skip it.

Stage 2 Something is going to happen but nothing can be done now. Means there are no slots to process the technical debts because the impact of Technical debt is not clear.

Stage 3 May be we should do something about it but nothing we can do at this moment. No budjet or slot to mobilize resources for closing Technical debts. Even though the Technical debt has some impact on IT ecosystem

Stage 4 Something we could have done but it’s too late.

I call this behavior procrastination

Oxford English Dictionary : Procastination: as a postponement, “often with the sense of deferring though indecision, when early action would have been preferable,” or as “defer[ing] action, especially without good reason.”

Result: Years OF Accrued Technical Debts

When such technical debts are applied in middle or later stages of digital journey, it creates slippery downward spiral syndrome. Slippery downward spiral means incumbents feel difficult to come out of technical debts like a person who is not able to repay his debts or mortgage on time. Feel is like painting a flying Aeroplane. Unplanned work increases.

When the digital journey is at the peak, new business features become 100 folds, technical debts become 100o folds. Business processes impacted by implementing technical debts increases by 10000 folds. Below are typical scenarios

Requirements form different areas in different tools like caliber or Jira, excels, powerpoint hooping handoffs. Some requirements getting dropped over time during so many case exchanges.

Ever increasing. demand of code checkins every minute, from different project teams or scrum teams developers. Not sure if code change of one team impact the other team. Need for a seamless code configuration tool to manage the code and executable complex management and dependencies is never complete. Adding to this complexity if there are more than one code management tools.

development teams deployment in Dev environment every hour. More than one projects at same time in DEV environment of a technology like to deploy concurrently . More than one suppliers are using the same DEV environment . This increases chance of stepping into each other shoes.

Deployment in TST every day. For 5 min deployment the governance and approvals takes almost 1 hr plus Testing and Business team need lot of time to generate test data for envisioned new features that are not yet live.

Testing in Test Environment and Acceptance environment every day. Testing done by one project team is not known to another. Environment Restart request of one team impacts the testing of others. Often 7 am – 7 pm working day seems less to complete testing end to end and evening TST deployment gets deferred.

Lengthy approval processes for the IT instances. lengthy non IT processes like procurement, legal, contract management further slows down the flow of features, technical debts, risks ans incident fixed to Production.

Complaince team constantly check the security and compliance controls. They kind of intrude into the Deep work time of technologists . Compliance team have all their good intention to secure the organization. They keep trying to find the balance between transparency, traceability and transition. I am not surprised, if Developers and IT staff are praying not to recieve any non- conformance. The processes and governance that IT staff gets audited gets outdated and distorted because of day to day entropy and chaos in the way of working to fulfill the software production at scale.

Agile Jira board are used to visualize the flow of features, incident fixes, technical debts and risks. Most of the time these Agile stories and Agile boards are not interlinked and Jira stories become more or less like project management tool. Whereas the principle behind Agile and DevOps tools framework like Jira was to bring minimum viable products, Go live as early as possible, foster experimentation, instead of IT project mindset bring in IT product mindset, increase the feature flow and through put plus Automate all the IT human tasks. But developers and technologists end up in updating this plethoras of Agile tools, excel sheets, status reports etc. Instead of focusing deeply on development Deepwork, they are busy in shallow work. Productivity of technology teams decreases drastically vs expectation from leadership is to increase their productivity. Habit of keeping lot of work in progress increases and many unfinished work reaches the subsequent work centers.

In today’s information Economy, technologies changes very often. Every month there is new technology release or next month the current version will become obsolete.These technology teams has to cope with the speed of technology change. Technology teams have to unlearn the old version and quickly adapt the new version. This vendor tools features and Infrastructure features development run in parallel to evolving busieness features. Literally there is fight to get the schedule to deploy these technology change and infra change . Lot of time is wasted in approvals and discussion. There is often Planned downtime of environments for technology life cycle management (pathing, upgrades etc) during non business hours . But there are always surprises, there can be Unplanned downtime because of hardware failures, network congestion, new firewall rules etc impacting the Business users in the middle of Demo and Testing. These unplanned downtime may be because one or more of the technical debts are not addressed on time. These technical debts can be as simple as a 5 min upgrade but they often loose the battle of winning slots in DEV, TST,ACC or PROD because of traditional IT mindset of approvals and governance or business features are higher priority than technical debts.

Inspite of all hurdles somehow many business features and few technical debts make their way to production deployment every day. However by the time Feedback from the production business users or technology teams reach the development team , the development teams are already busy with new releases. Often transition of new live features and live technical debts in PROD is done operational teams. But their learning curve is too slow to resolve the incidents. New operations team finds it difficult to adopt to the new Digitized IT.

Within one or two years of Digital journey, incumbents IT Department realize they have tons of features,defects and technical debts all fighting to gain slot in lower environments for development, testing and Go lives. Some thing like Indian traffic jam.

Picture Source indiagoodnews.org

Each feature, defect, technical debts and risks all want to Go at the same time. 24 hrs in a day seems less. Weekend and Non business hrs seems less to manage processing of features, defects, technical debts and risks all the same time. Not only time but unplanned work grows and so does the unplanned cost.

Below picture is ideal picture to depict the Time Crunch, pressure and sense of urgency.

For Tech giants digital firms (like Amazon,Google) and digital startups , it is surely much easier to setup a closed-loop system to experiment and such downward spiral never occurs. It is much harder when you are traditional brick-and-mortar business. However, when you think of it, it is not that much harder also! We have a plethora of A/B testing and deployment methods, and advancements in technology that could be used to prototype and test alternate hypotheses and reduce technical debts. Extend the experimentation to technical debts along with new business feature. Don’t limit the experimentation to busienss features only.

Bigger issue lies is the mindset and a culture that discourage the experimentations. There should be regular feedback loop to make businesses aware about the technical debts inventory. Business should be aware that if the technical debts are not addressed on time, It will have serious economical consequences on business only . There is need of Business and IT Enterprise Architecture that allow incumbents to build Software solutions at scale to meet the Digital strategy and increase Sales and Revenues. At last I want to quote the below for incumbents currently embarking on digital transformation journey:

“People try to copy Netflix, but they can only copy what they see. They copy the results, not the process.” — ADRIAN COCKCROFT, FORMER NETFLIX CHIEF CLOUD ARCHITECT