Next-Gen
ADM Services
A research report comparing provider strengths,
challenges and competitive differentiators
Customized report courtesy of:

QUADRANT REPORT—| —SEPTEMBER 2023 —| —U.S.

Table of Contents

Executive Summary03
Provider Positioning07
Introduction
Definition12
Scope of Report
13
Provider Classifications
14

Appendix
Methodology & Team
Author & Editor Biographies
About Our Company & Research

47
48
50

Agile Application
Development
OutsourcingWho Should Read This Section 
Quadrant 
Definition & Eligibility Criteria
Observations
Provider Profile

Agile Application
Development
Projects

15 – 20
16
17
18
19
20

Continuous TestingSpecialists

Who Should Read This Section 
Quadrant 
Definition & Eligibility Criteria
Observations
Provider ProfileWho Should Read This Section
Quadrant
Definition & Eligibility Criteria
Observations

Application Quality
Assurance
21 – 25

Who Should Read This Section 
22
Quadrant 
23Definition & Eligibility Criteria
24
Observations25© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Application Managed
26 – 32
Services

Who Should Read This Section 
Quadrant 
Definition & Eligibility Criteria
Observations
Provider Profile

27
28
29
30
32

41
42
43
44

33 – 39
34
35
36
37
39

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

40 – 452

t

Executive Summary

Report Author: Akhila Harinarayan

DevOps and SRE lead the way for application
development and maintenance with AI
support
The application development and maintenance(ADM) domain has undergone multiple changes
over the last few years. These changes havebeen fast paced compared to the previous
generations of application development. With
organizations undergoing digital transformation,
speed, cost and agility play a vital role in ADM.
Enterprises seek partners that can help themrealize the potential of digital innovation.
Service providers have embedded this aspectin their digital offerings that are aligned withthe most advanced ADM methodologies.
The graphic below showcases the periodic
changes in the ADM space.
With the advent of new approaches andmethodologies, there is a significant shift in the
adoption of Agile and DevOps methodologies
for application development. Increased
emphasis is placed on developing applicationsaligned to enterprises’ digital journey using
Agile and DevOps methods. This engagement
enables faster time-to-market, enhanced
collaboration and improved quality. It alsoenables an innovative approach to delivering
applications aligned with specific business
requirements. With businesses’ economic
constraints, cost optimization has become akey focus area. Application maintenance has
witnessed the use of technologies, such as
automation, analytics and AI, to optimize theentire app maintenance process and reduce
human intervention, delivering cost savings
that can fuel new initiatives in app development
for enterprises. Site reliability engineering(SRE) adoption has contributed to enhanced
reliability, predictable operations, performancemeasurement and qualitative applications
development. DevOps and SRE act as balancing
factors to deliver a high-quality application.
DevOps enables the disintegration of traditionalsilos into development and operations toimprove the efficiency and reliability of softwaredevelopment and deployment processes.
SRE focuses on creating highly scalable andreliable software systems while increasingly
emphasizing automation and monitoring.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Generative AI
assists developers
by generatingcode and isincreasingly in use.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

3

t

Executive Summary

ADM Evolution over years
Application
Development

Application
Maintenance Developers worked directly

Client service model with
Integrated Development
Environments (IDEs)
providing tools to aid
development.

Application development
focused on business
requirements and flexible
architecture for code reuseApplication development
integrates with testing andassessing if the application
quality is as expected including
DevOps, SRE and low-code,
no code appsOutsourcing of maintenanceservices also became moreprevalent. It included
enhancements and updates
as well as maintaining
applications

 Automation and analytics-basedIntuitive approach to assess
and analyze improvements
across business processes
including use of AI,
automation, low-codewith machine languages
and assembly languages tocreate software - manual
coding and programmingt

Application maintenanceprimarily involved manual
debugging, fixing defects,
and applying patches

Standalone
solutions

Client-server
Waterfall model

Generation 1
1960s-90s

Generation 2
1990s-2000s© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

application maintenance todeliver tangible benefits toclients, such as ensuring
availability, performance, andsecurity of online services

API, Microservices
Agile methods

DevOps and SRE
AI-based ADM

Generation 3

Generation 4

2000s-2020s

2020s-Present

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

4

t

Executive Summary

Hence, DevOps and SRE are crucial in
optimizing software development, deployment
and operations processes. They contribute tothe overall efficiency, reliability and success ofdigital products and services.
The increased adoption of AI and generative AI
has a significant impact on the ADM lifecycle.
Some of the use cases include the following
tools: AI-driven predictive analytics thataids project planning by predicting resource
requirements and potential bottlenecks;
AI-driven design tools that generate userinterfaces, layouts and prototypes based on user
requirements and design principles; AI-powered
testing tools that automate test case generation,
increase test coverage and help quickly detect
defects; AIOps platforms that use AI to monitor
and manage application performance, predictand prevent downtime and optimize resource
usage; predictive maintenance powered by
AI that analyzes historical data to anticipate
maintenance needs and proactively schedule
updates; and AI-driven analytics that provides
insights into application performance, userinteractions and usage patterns, contributing tocontinuous improvements. Generative AI canassist developers by generating code snippets
based on natural language descriptions orcompleting code lines. While AI has significantly
impacted the ADM lifecycle, the generative AI
impact is not entirely proven across the use
cases. With these changes in the industry,
the workforce plans to combine humans andmachines. A combination of bots that performroutine tasks and cognitive functions andencompass analytical abilities will assist thehuman workforce.
In addition to these technical developments,
there is an increased focus on establishing
global capability centers (GCCs) that support
businesses in the U.S. These GCCs are typicallyestablished in India, eastern Europe or Latin
America. GCCs established in India had atremendous increase in delivering innovativeand cutting-edge applications and products
developed in collaboration with service
provider partners.
Top trends in ADM segments are listed below.
Agile application development outsourcing:
The use of AI across application development
and business-led, cloud-based application© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

development are two major trends observedin this segment. Enterprises seek application
development with an AI component intheir contracts. They focus on transforming
monolithic applications by incorporating theminto cloud architecture. This process requires
significant investments in upskilling the existing
talent pool. Most application development isdriven by enterprises’ digital agendas, with anincreased emphasis on delivering application
development through Agile, DevOps and
SRE aligned with digital product-oriented
development (POD) models. Some of theunique contract models being used include
experience-level agreements mapped tobusiness imperatives.
Agile application development projects:
Enterprises focus on CX and plan to emphasize
the delivery of exceptional UX. They increasingly
embrace Agile methodologies and DevOps
practices to accelerate software development
and enhance collaboration between
development and operations teams. Agileand DevOps enable faster time-to-market,
improved quality and increased flexibility inresponding to changing business needs.

Enterprises increasingly focus on cloud
engagements and infrastructure modernization.
They further concentrate on transitioning theirapplications to cloud-native architectures,leveraging containerization and microservices.
This shift allows for greater scalability,
resilience and agility in deploying andmanaging applications.
Application managed services:
Most enterprises attempt to optimize costand efficiency in managing applications
within their IT landscape. Service providers
have devised methodologies and approaches
to use technologies such as AI, automation
and analytics to deliver tangible benefits
to clients. Data-driven approaches deliver
better experiences and adhere to the agreedupon KPIs. As cloud adoption increases,
the need to manage cloud applications
and optimize infrastructure availability
and application performance becomes
essential. Service providers align with market
expectations and leverage AI — to a certain
extent, generative AI — to deliver application
managed services to their clients.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

5

t

Executive Summary

Application quality assurance: In today’s
rapidly evolving development landscape,organizations face shorter software release
cycles, necessitating customized testing
solutions and adopting DevOps practices andtools. Service providers’ quality assurance (QA)
practices focus on achieving exceptional UX.
Providers also emphasize the developmentof cloud-based automation platforms thatleverage AI and ML to address enterprisedemands for faster cycles. These testing
platforms use codeless, self-healing andpredictive test automation to enable afaster and more efficient software delivery.
The increased demand for shift-left testing
approaches using AI and ML for test
automation has helped improve application
quality. Using SRE to improve application
quality before deployment into the production
environment, thus reducing application
downtime, is also an emerging trend.

This engagement enables enterprises tosimulate real-world scenarios and perform
comprehensive testing across different
platforms. With the intense pressure todeliver efficient applications, leveraging AI
and automation has become a mandate forservice providers. This setup enables providers
to use AI-powered testing and automation to
help clients identify patterns, predict potential
defects and optimize test cases. Advanced
generative AI techniques are used for AI-led
software testing. Firms prioritize security
testing and compliance owing to the increasing
number of cyber threats and data breaches.
They incorporate robust security testing
practices, including vulnerability assessments,
penetration testing and code analysis.

AI has a significant impact acrossthe ADM lifecycle, reducing thedelivery time and improving thequality of applications delivered.
As the industry is moving toward
reducing the time required
for testing, there is increased
traction on adopting DevOps and
SRE for ADM.

Continuous testing specialists:
Enterprises focus on leveraging cloud-based
testing and ADM services to enhance
scalability, flexibility and cost-effectiveness as
the adoption of cloud applications increases.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

6

tProvider Positioning

Provider Positioning

Page 1 of 5

Agile Application
Development Outsourcing

Agile Application
Development Projects

Application Managed
Services

Application Quality
AssuranceContinuous TestingSpecialists

a1qa

Not InNot InNot InContender

Not InAccenture

LeaderNot InLeaderLeaderNot InApexonNot InNot InNot InNot InLeaderAspire Systems

Not InContender

Not InContender

Not InBirlasoft

Contender

Not InContender

Not InLeaderCapgemini

LeaderNot InLeaderLeaderNot InCignitiNot InNot InNot InNot InLeader

Coforge

Contender

Not InProduct Challenger

Not InMarket Challenger

CognizantLeaderNot InLeaderLeaderNot InConcentrix

Not InContender

Not InNot InContender© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

7

tProvider Positioning

Provider Positioning

Page 2 of 5

Agile Application
Development Outsourcing

Agile Application
Development Projects

Application Managed
Services

Application Quality
AssuranceContinuous TestingSpecialists

CybageNot InLeaderContender

Not InProduct Challenger

Deloitte

LeaderNot InLeaderLeaderNot InDXC Technology

Market Challenger

Not InLeaderProduct Challenger

Not Ine-coreNot InContender

Not InNot InNot InEncoraNot InLeaderContender

Product Challenger

Not InEvidenProduct Challenger

Not InProduct Challenger

Not InLeaderFujitsu

Not InNot InNot InProduct Challenger

Not InHCLTechLeaderNot InLeaderLeaderNot InHexawareNot InProduct Challenger

Product Challenger

Not InLeaderHTC Global

Not InLeaderProduct Challenger

Not InProduct Challenger© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

8

tProvider Positioning

Provider Positioning

Page 3 of 5

Agile Application
Development Outsourcing

Agile Application
Development Projects

Application Managed
Services

Application Quality
AssuranceContinuous TestingSpecialists

IBM

Contender

Not InContender

Contender

Not InIndiumNot InNot InNot InContender

Not InInfinite

Not InMarket Challenger

Contender

Product Challenger

Not InInfosys

LeaderNot InLeaderLeaderNot InInnominds

Not InContender

Not InContender

Not InITC Infotech

Not InProduct Challenger

Not InNot InContender

IterisNot InContender

Not InNot InNot InKyndryl

Product Challenger

Not InProduct Challenger

Product Challenger

Not InLTIMindtree

Product Challenger

Not InLeaderRising Star

Not InMarlabs

Contender

Not InContender

Not InContender© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

9

tProvider Positioning

Provider Positioning

Page 4 of 5

Agile Application
Development Outsourcing

Agile Application
Development Projects

Application Managed
Services

Application Quality
AssuranceContinuous TestingSpecialists

MastekNot InProduct Challenger

Not InNot InNot InMphasisContender

Not InContender

Contender

Not InN-iX

Not InProduct Challenger

Not InNot InContender

NTT DATA

Contender

Not InNot InNot InNot InPersistent Systems

Contender

Not InProduct Challenger

Not InProduct Challenger

QA Consultants

Not InNot InNot InProduct Challenger

Not InQualitest

Not InNot InNot InNot InLeaderQuinnox

Not InProduct Challenger

Rising Star

Contender

Not InSLK Software

Not InContender

Contender

Contender

Not InSofttek

Product Challenger

Not InContender

Not InContender© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

10

tProvider Positioning

Provider Positioning

Page 5 of 5

Agile Application
Development Outsourcing

Agile Application
Development Projects

Application Managed
Services

Application Quality
AssuranceContinuous TestingSpecialists

TCS

LeaderNot InLeaderLeaderNot InTech Mahindra

Product Challenger

Not InLeaderNot InLeaderTestingXperts

Not InNot InNot InNot InContender

Trigent Software

Not InProduct Challenger

Not InContender

Not InUST

Not InLeaderContender

Not InLeader

Wipro

LeaderNot InLeaderLeaderNot InZensarNot InLeaderContender

Not InProduct Challenger© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

11

Introduction

Definition

Five quadrants
cover the key
capabilities
in planning,development,
quality control
and deploymentof software
applications.
Simplified Illustration; Source: ISG 2023

Agile Application Development
Outsourcing

Agile Application Development
Projects

Application Managed Services

Application Quality Assurance

Leveraging software capabilities tointegrate all business layers, create newdata sources and gain enterprise agility is
an indispensable requirement for modern
application outsourcing.
Next-gen ADM services include consulting,design, custom development, packaged
software integration, application management
and operations, quality assurance, security
services and testing.
Cloud-based computing and the rising demand
for automation and AI drive the market forcloud-native application development and giveit a new focus. Service providers emphasize
Agile methodologies and the continuous,secure delivery and automation of softwaredevelopment processes with DevSecOps,
Tailor-made roadmaps combine digital,
operational and technology goals to meetclients’ objectives.

Service providers enable organizations toautomate routine tasks and gain deeper
insights into their application development
processes using AI. This has led to thedevelopment of new tools and platforms thatincorporate automation and AI capabilities toaccelerate development cycles; ensure security,
threat detection and vulnerability management;
and improve end-user experience; this, inturn, helps deliver intuitive, engaging andpersonalized applications.
This study focuses on the recent developmentsthat have taken place across the application
development, application management andquality assurance markets. Simultaneously,
ISG is launching the 2023 ISG Provider Lens™
Next-Gen ADM Solutions - Low-Code/No-Code
Development Platforms 2023 study to offer
clients a broader understanding of theapplication solutions market.

Continuous Testing Specialists© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

12

Introduction

Scope of the Report

Provider Classifications

This ISG Provider Lens™ quadrant reportcovers the following five quadrants forservices: Agile Application Development
Outsourcing; Agile Application Development
Projects; Application Managed Services;
Application Quality Assurance; and Continuous
Testing Specialists.

The provider position reflects the suitabilityof IT providers for a defined market segment(quadrant). Without further additions, theposition always applies to all company sizes
classes and industries. In case the IT service
requirements from enterprise customers differ
and the spectrum of IT providers operatingin the local market is sufficiently wide, afurther differentiation of the IT providers byperformance is made according to the target
group for products and services. In doing so,
ISG either considers the industry requirementsor the number of employees, as well as
the corporate structures of customers andpositions IT providers according to their focus
area. As a result, ISG differentiates them, if
necessary, into two client target groups that are
defined as follows:

The ISG Provider Lens™ Next-Gen ADM
Services 2023 study offers the following tobusinesses and IT decision-makers:● Transparency on the strengths andweaknesses of relevant providers● A differentiated positioning of providers bysegments on their competitive strengths andportfolio attractiveness● Focus on regional market
ISG studies serve as an important decisionmaking basis for positioning, key relationships
and go-to-market considerations. ISG advisorsand enterprise clients also use information fromthese reports to evaluate their current vendor
relationships and potential engagements.

● Midmarket: Companies with 100 to 4,999
employees or revenues between $20 million
and $999 million with central headquarters
in the respective country, usually privately
owned.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

● Large Accounts: Multinational companies
with more than 5,000 employees or revenue
above $1 billion, with activities worldwide
and globally distributed decision-making
structures.
The ISG Provider Lens™ quadrants are createdusing an evaluation matrix containing four
segments (Leader, Product Challenger, Market
Challenger and Contender), and the providers
are positioned accordingly. Each ISG Provider
Lens™ quadrant may include service providers
that ISG believes have strong potential to moveinto the Leader quadrant. This type of provider
can be classified as a Rising Star.
● Number of providers in each quadrant:
ISG rates and positions the most relevant
providers according to the scope of thereport for each quadrant and limits the
maximum of providers per quadrant to
25 (exceptions are possible).

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

13

Introduction

Provider Classifications: Quadrant Key

Product Challengers offer a product andservice portfolio that reflect excellent service
and technology stacks. These providers andvendors deliver an unmatched broad and deep
range of capabilities. They show evidence ofinvesting to enhance their market presence
and competitive strengths.

Leaders have a comprehensive product andservice offering, a strong market presence andestablished competitive position. The productportfolios and competitive strategies of Leaders
are strongly positioned to win business in themarkets covered by the study. The Leaders also
represent innovative strength and competitive
stability.

Contenders offer services and products
meeting the evaluation criteria that qualifies
them to be included in the IPL quadrant. These
promising service providers or vendors show
evidence of rapidly investing in products/
services and a follow sensible market approachwith a goal of becoming a Product or Market
Challenger within 12 to 18 months.

Market Challengers have a strong presencein the market and offer a significant edge
over other vendors and providers basedon competitive strength. Often, Market
Challengers are the established and well-known
vendors in the regions or vertical markets
covered in the study.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Rising Stars have promising
portfolios or the market experience
to become a Leader, including therequired roadmap and adequate
focus on key market trends
and customer requirements.
Rising Stars also have excellent
management and understandingof the local market in the studied
region. These vendors andservice providers give evidence ofsignificant progress toward theirgoals in the last 12 months. ISG
expects Rising Stars to reach theLeader quadrant within the next 12
to 24 months if they continue theirdelivery of above-average market
impact and strength of innovation.

Not in means the service provider
or vendor was not included in thisquadrant. Among the possible
reasons for this designation:
ISG could not obtain enoughinformation to position thecompany; the company does
not provide the relevant service
or solution as defined for eachquadrant of a study; or the company
did not meet the eligibility criteria
for the study quadrant. Omission
from the quadrant does not imply
that the service provider or vendor
does not offer or plan to offer thisservice or solution.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

14

avv

Agile Application Development
Outsourcing

Agile Application Development Outsourcing

Who Should Read This Section
This report is relevant to enterprises acrossmultiple industries in the U.S. for evaluating
providers that offer Agile application
development outsourcing services.
In this quadrant, ISG highlights the current
market positioning of providers offering Agileapplication development outsourcing services
in the U.S. based on the depth of their service
offerings and market presence.
Enterprises in the U.S. are investing timeand effort in digital transformation. They arefocusing on increasing automation using newtechnologies such as AI and ML to modernize
legacy systems. Modernization will bring moreautonomy and hyperinnovation to generatehigher business value. Enterprises focus onlegacy modernization, including rehosting,migration and rearchitecting for the cloud,reflecting the ongoing need to update andmodernize the existing systems for greater
efficiency and scalability. The emphasis on
cloud-based development, microservices

and containerization and the adoption of
low-code solutions suggest a broader trend ofmodernizing applications and infrastructure forscalability and agility. These trends drive thedemand for Agile app development services
in the region.
Agile outsourcing offers resource flexibility,
specialized skills, quick app releases, cost
reduction by focusing on core functions
and shortened time-to-market. As leading
tech providers witness high demand, service
providers invest in tools, frameworks andaccelerators for the ADM lifecycle to satisfy
enterprise requirements.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

IT professionals should read this report
to determine service providers’ strengths
and weaknesses in ADM and learn how
to integrate cutting-edge technologies formarket advantage.

Operations professionals should read thisreport to understand how service providers
can help generate more business value
while simplifying business operations
and processes.

Business professionals should review
this report to understand partner
positioning for efficient application service
procurement and favorable ROI in theirbusiness or industry.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

16

Source: ISG RESEARCH

Next-Gen ADM Services
Agile Application Development Outsourcing

U.S. 2023

H

Capgemini

PORTFOLIO ATTRACTIVENESS

Accenture

L

TCS

Infosys
Kyndryl

LTIMindtree
EvidenProduct Challenger

Softtek

Deloitte

Wipro

HCLTechLeaderTech Mahindra

Persistent Systems
IBM
NTT DATA
MphasisCoforge

Contender

Cognizant

DXC Technology

Market Challenger

This quadrant
assesses service
providers that offer
ADM expertise using
different technologies,
spanning the complete
application development
and management
landscape and mostindustry verticals.
Akhila Harinarayan

Birlasoft
Marlabs

L

COMPETITIVE STRENGTH© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

H

Rising Star

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

17

Agile Application Development Outsourcing

Definition
This quadrant assesses service providers
that offer ADM expertise with the use ofdifferent technologies, spanning the complete
application development and management
landscape and most industry verticals, inoutsourcing deals that are based on thedelivery capacity for a specific time frame(three- to five-year contracts, renewable).
ADM outsourcing offers capacities, regardlessof the number and size of projects andprogramming languages, to support clients’application portfolios or business units.
It covers large and highly complex application
landscapes that can span multiple geographic
locations, lines of businesses and organizational
entities. One of its roles is to break up silos
in both organization and technologies,
create unified technology platforms for theapplication development landscape, and thus
allow faster and more innovative go-to-market
on a large scale.

This study evaluates how service providers
use project management tools, platform as aservice (PaaS), software as a service (SaaS),
low-code/no-code platforms or otheraccelerators to elevate clients’ application
development and management capabilities.
A typical service provider in this quadrant has
extensive consulting expertise and high-end
technology partnerships to implement CI/CD
pipelines, application testing and DevOps toenable clients to achieve high performance
while reducing time to market.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

prioritization, Agile methods,
Waterfall methods, system
integration, application
modernization, cloud-native
application development andother services to optimize
development teams

Eligibility Criteria

1.

Management of more than
20 squads for a single clientor being able to scale up to
more than 1,000 developers,
working simultaneously onseveral projects

2.

Ability to rapidly scale up
or down and add more than
100 developers in a week tomeet the demands of a client as
necessary

3.

Comprehensive Application
Development Platform thatcovers resource allocation,
portfolio management, backlog

4. Certified to transform and deploy
Agile teams under frameworks
such as Scaled Agile Framework(SAFe) and Large-Scale
Scrum (LeSS)

5.

Certified experts in Scrum,
Kanban, Lean development orother Agile methodologies

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

18

Agile Application Development Outsourcing

Observations
Two prominent trends have emerged as
powerful catalysts for innovation andtransformation in application development
outsourcing for large enterprises. The first trend
involves the pervasive use of AI throughout
the development process. Enterprises are notmerely considering AI as an option; instead,they consider it as an integral component
within their development contracts. This
engagement reflects a strategic shift towardharnessing AI’s capabilities to enhance
application functionality, UX and efficiency.
The second trend revolves around cloudbased application development driven
by business imperatives. Enterprises are
strategically migrating from traditionalmonolithic applications to dynamic cloudarchitectures. This transformation necessitates
substantial investments in upgrading theskills of the existing workforce to navigate thenuances of cloud environments effectively.
Cost optimization has been a major factor
for enterprises, and service providers havehelped clients through various approaches,
including low-code and no-code applications

and increased focus on establishing global
capability and microcapability centers.
Optimizing managed application services todrive savings for new application development
initiatives is also an approach service providers
undertake to expand their footprint within theirclient base and across the industry/region.
From the 86 companies assessed for thisstudy, 21 qualified for this quadrant, with eight
being Leaders.

Accenture has many tools, frameworks andaccelerators that help clients across the ADM
lifecycle. The company’s robust understandingof clients’ businesses and industry segments
helps it apply these to realize benefits for
its clients.

Capgemini offers ADMnext, a global, unified
approach to apply Agile, DevOps, cloud, API,data platforms and security at an industrial
scale in the IT department. This approachentirely embraces the migration toward
product-centricity.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Cognizant has developed more than 100 IP
assets covering hybrid cloud integration,
API management and microservices to drive
efficiency and enable faster time-to-market.

Deloitte combines Agile methods and productmindset to deliver value-driven solutions.
With platforms such as DevOps Cloudand innovative tools such as TurboCode,it exemplifies excellence in driving digitaltransformations and achieving superiorbusiness outcomes.

HCLTech offers out-of-the-box AI and ML
platforms with no-code/low-code functionalities
for data scientists and non-data scientists
to create guided and guard-railed solutions,along with the ability for quick deployment,
monitoring, governance and maintenance.

Infosys AI and Automation Services
follows a product-agnostic approach andhas expertise in operating most of theleading AI platforms, including open-source
technologies. The company has developed
tools and accelerators to help accelerateclients’ AI journeys.

TCS continues to invest in training, reskillingand upskilling its workforce to meet theevolving client demands and expanding the
adoption of Agile methodologies. The company
has a robust culture of continuous learning andtalent upskilling.

Wipro has high-level partnerships withleading hyperscalers such as Amazon, Google
and Microsoft. The company is investing
significantly in startups with expertise in
Agile, cloud technologies and DevOps, among
other areas.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

19

Agile Application Development Outsourcing
LeaderCognizant“Cognizant has developed new reference models
for modern delivery, architecture and talentmanagement and aligned software development
capabilities with 55 industry offerings.”
Akhila Harinarayan

Overview

Strengths

Cognizant is headquartered in New
Jersey, U.S. and operates in 42 countries.
It has more than 351,500 employees
across 162 global offices. In FY22 thecompany generated $19.4 billion inrevenue, with Financial Services as
its largest segment. The Cognizant
application development practice ishoned and nurtured under Cognizant’s
Software and Platform Engineering(SPE) Unit, which brings together
offerings across software, data,
platforms, industries and the cloud.
Cognizant offers services through threestrategic C-suite narratives for digitalproduct development, application
modernization and cost to value.

Modern stack talent: Cognizant has arobust talent pool across varied technologies
ranging from legacy to next-generation
digital technology stack. It is building
digital skills across full stack engineering(FSE), DevSecOps, SRE, containerization,
Kubernetes and hybrid cloud management
platforms, cloud/digital architecture andperformance engineering, and microservices
and API management.
Data science and AI and ML: Cognizant’s
data science centers of excellence (CoEs)
have been helping clients develop andbuild AI and ML models. It has developed aminimum viable product (MVP)-based Agile
approach for clients to deliver innovative data
science solutions using open-source toolkits(Python/R/Github/Jupyter notebooks) topredict asset failure risks.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Vital assets for transformations:
Cognizant has developed more than 100 IP
assets covering hybrid cloud integration, API
management and microservices for driving
efficiency and enabling faster time-to-market.
It has launched Cognizant Skygrade, amulti-hybrid cloud and edge management
platform designed to help firms transition
to modern cloud-native architectures andstreamline cloud management operations.
The platform operates seamlessly acrossmulticloud environments, simplifying
complex cloud management, enabling clientsto operate cloud-native businesses andaddressing cloud transformation lifecycle.

CautionCognizant should focus on improving themessaging around AI use across the ADM
lifecycle. Most competitors are focused ondelivering benefits to clients by using AI to
reduce the entire lifecycle time.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

20

avv

Agile Application
Development Projects

Agile Application Development Projects

Who Should Read This Section
This report is relevant to enterprises acrossindustries in the U.S. for evaluating providers
that offer services for Agile application
development projects. These engagements
typically have restricted or limited delivery
scope, and the requirement for squaddeployment is less than 20.
In this quadrant, ISG highlights the current
market positioning of providers that offer Agileapplication development project services in
the U.S. based on the depth of their service
offerings and market presence.
Enterprises prioritize quick project turnaroundand exceptional CX, supporting the growing
demand for advanced digital technologies, suchas AI, ML and low-code/no-code platforms.
AI and ML adoption helps enterprises increaseautomation by creating more business value.
Low-code, no-code application development
is on the rise, and this trend is driven by
the demand for rapid innovation and agility
in application development, especially forsmaller-footprint applications. Enterprises seek
service providers that offer Agile application

development and have expertise in newtechnologies and strong partnerships.
These providers should also be able toexpand their capabilities by investing in skill
development and training.
Businesses are focusing on delivering seamless
interactions through Agile methodologies
and DevOps practices, driving cloud-based
ADM growth and infrastructure modernization
services. Cost-effectiveness, scalability andsecurity are crucial considerations. As digitaltransformation drives business growth,
providers must scale quickly, integrate NextGen
technologies, and demonstrate proven security,
scalability and integrity track records.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

IT professionals should read this report
to determine service providers’ strengths
and weaknesses in ADM and learn how
to integrate cutting-edge technologies formarket advantage.

Operations professionals should read thisreport to understand how service providers
can help generate more business value
while simplifying business operations
and processes.

Business professionals should review
this report to understand partner
positioning for efficient application service
procurement and favorable ROI in theirbusiness or industry.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

22

Source: ISG RESEARCH

Next-Gen ADM Services
Agile Application Development Projects

U.S. 2023

PORTFOLIO ATTRACTIVENESS

H

Quinnox

Hexaware
ITC Infotech

Mastek

Product Challenger

Trigent Software
Contender

SLK Software

Innominds

N-iX

UST

CybageHTC Global
EncoraLeaderZensarConcentrix

Infinite
Market Challenger

This quadrant evaluates
service providers
that deliver Agileapplication development
for projects withwell-defined scopes,
specific application
development expertise,business objectives
or squad-based
capabilities.
Akhila Harinarayan

Aspire Systems
e-coreIterisL

L

COMPETITIVE STRENGTH© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

H

Rising Star

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

23

Agile Application Development Projects

Definition
This quadrant evaluates service providers thatoffer Agile application development in deals
that include clear scope definitions for project
outcomes, specific application development
skills, business goals or squad capacity.
Typical service providers in this quadrant
offer expertise to ensure successful businessoutcomes for each Agile project. Deals can
include a fixed number of team members
per squad or flexible models measured byapplication feature delivery or other pricing
methods. Project engagements can vary fromsmall mobile applications to large solution
implementations. Typical engagements
take less than 18 months. Large projects are
exceptions and most likely have staggered
releases or more sprints.

Service providers in this quadrant are also
responsible for the full management of theirdelivery teams. The ability to engage manysquads to support a client is considered, but
application staff augmentation services are
excluded from this quadrant. Providers should
manage the squad size and offer experts
according to throughput targets.
These service providers add specificknowledge and skills required by squads or
projects and can differentiate themselves byoffering business expertise; highly specialized
expertise in dedicated industries, businessenvironments or technology areas; or expertisein development accelerators. The commercial
business model centers on the provisionof squads for client-managed application
development units.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

3.

Eligibility Criteria

1.

2.

Projects are typically measured
by the number of squadmembers, user stories delivered,deployment rate/frequency,
defect count, time to market andbusiness-related indicators, suchas shared business outcomes
Certified experts in Agile
methods, such as Scrum, Kanban
or Lean development; cloudnative data analytics; low-code/
no-code development; systemarchitecture; and CX design

Proof-of-delivery capacity withclient references; they should
not be startups or recently
established companies

4. Talent acquisition programs,

training programs, knowledge
management processes andthe provision of a healthy work
environment to retain top talent

5.

Business expertise ordevelopment accelerators
for CRM, e-commerce, ERP or
industry-specific technologies

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

24

Agile Application Development Projects

Observations
Enterprises have shifted their strategic focus
to prioritize exceptional CX. Recognizing thatuser satisfaction is a critical differentiator,
businesses are directing their efforts toward
delivering seamless and engaging interactions.
They are embracing Agile methodologies and
DevOps practices, revolutionizing the softwaredevelopment approach.
Cloud-based ADM offerings have experienced
rapid growth due to their transformative
benefits. Service providers focus onbuilding offerings and skillsets/talent tocater to their clients’ increasing demands.
Cloud transformation and infrastructure
modernizations are the most sought-after
services by providers focused on building value
propositions and positioning their tools andaccelerators as differentiators.

development is focused on targeting the end
goal while delivering qualitative applications
at a rapid pace. These trends are expected
to continue, with more and more enterprises
opting to undergo digital transformation to
grow their businesses and providers delivering
these services through differentiated offeringsand partner networks across the region.
From the 86 companies assessed for thisstudy, 18 qualified for this quadrant, with fivebeing Leaders.

Cybage excels in diverse ADM streams,
offering skilled staff for UX, COTS integrationand frontend/backend services. Its Product
Intensive Engineering (PIE) frameworkensures quality, Agile development and fastertime-to-market.

Encora’s extended delivery network includes
one innovation lab and CoEs in the U.S. anddelivery centers in the Americas, ensuring
cultural affinity, rapid feedback and cost
efficiency. It has a robust partner network
worldwide that uses advanced technologies.

HTC Global Services
HTC Global Services excels in DevSecOps
consulting, web-based enterprise app
integration, COTS product integration, ERP
systems implementation, and AI- and ML-driven
innovations for process automation, fault
prediction and personalized solutions.

UST has established skills and committed
assets in ADM fields such as COTS productintegration, DevSecOps consulting and UX.
Access to platforms, tools, accelerators andother IP enhances developer productivity,
SDLC visibility, standardization and SRE
model efficiency.

Zensar focuses on CX, microservices, API,
cloud-native and legacy modernization.
It invests in platforms such as ZIIAS for Agiletransformation. Trained resources, specialized
frameworks and accelerators ensure
DevSecOps and faster time-to-market.

Providers with ready PoCs that can help theirclients quickly enter the market will stand out
from competitors. Cost-effectiveness is alsoa major aspect of consideration. With clientsseeking multiple benefits for their IT landscapeand business requirements, application© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

25

avv

Application Managed Services

Application Managed Services

Who Should Read This Section
This report is for U.S. enterprises that areevaluating application managed service
providers. ISG’s quadrant report outlines
current market positions and evaluates how
providers overcome challenges clients face.
The rising demand for app modernization,
automation and value fuels the need for
application managed services in the U.S.
DevOps, DevSecOps and AIOps address
cost and speed priorities. Hybrid systems
and platform-independent application
complexity drive the managed service demand.
Clients seek L2/L3 support, security, incident
management and maintenance. The local
presence of providers is favored despite theremote work trend.
Service providers strategically leverage
AI, automation and advanced analytics tooptimize cost and operational efficiency while
ensuring exceptional UX and adherence topredefined KPIs. Cloud infrastructure adoptionemphasizes the need for proficient application
management, including infrastructure
optimization and sustaining its performance.

Service providers have harnessed AI,
including generative AI, to proactively identify
performance issues and vulnerabilities, thereby
enabling preemptive measures and continuous
operation. Outcome-based pricing models,aligned with KPIs and targets, benefit bothproviders and clients.
Providers excel with a KPI-focused strategy,
employing AI tools and assets to boost
automation, agility and quality. Theyprioritize cutting costs, achieving businessgoals and enhancing CX through inventive
intellectual property. The emphasis is on
predictive operations, employing AI-driven
analytics, ML algorithms, accelerators andchatbots. Their services include intelligent
operations, user-centered experiences and aplatform-based method for achieving value.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

IT professionals should read this report
to determine service providers’ strengths
and weaknesses in ADM and learn how
to integrate cutting-edge technologies formarket advantage.

Procurement managers should read thisreport to understand the service provider
ecosystem for application maintenanceservices in the U.S. and assess how various
providers compare with each other.

Business professionals should review
this report to understand partner
positioning for efficient application service
procurement and favorable ROI in theirbusiness or industry.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

27

Source: ISG RESEARCH

Next-Gen ADM Services
Application Managed Services

U.S. 2023

PORTFOLIO ATTRACTIVENESS

H

HexawareQuinnox
Coforge

Kyndryl
EvidenPersistent Systems
HTC Global
Birlasoft
EncoraUST
Infinite
CybageMphasisMarlabs
SofftekIBM
ZensarProduct Challenger

Contender

LTIMindtree
DXC Technology
HCLTechCapgemini
CognizantAccenture

Infosys
Wipro TCS
Deloitte
Tech Mahindra

LeaderMarket Challenger

The quadrant evaluatesproviders responsible
for managing enterpriseapplications in
production. Services
comprise application
support, enhancements,
platform upgrades,
security, bug fixing,
troubleshooting andmerging enhancements,
among others.
Akhila Harinarayan

SLK Software

L

L

COMPETITIVE STRENGTH© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

H

Rising Star

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

28

Application Managed Services

Definition
This quadrant assesses service providers
responsible for managing clients’ defined
application portfolios (applications in
production). It does not include nicheapplication specialists. Application managed
services (AMS) comprise application support,enhancements, platform upgrades, application
security, bug fixing, troubleshooting, andmerging enhancements and development
backlogs under Kanban or similar
methodologies. The leading service providers
in this quadrant offer application monitoring,
release management, version control, defectidentification, and resolution and database
query performance.

Service transition and client onboarding should
include application documentation, service
ticket records, knowledge transfer and expert
transfer/hire optionally. Continuous service
delivery starts after the transition period
ends and often includes quality improvement
programs and service knowledge refresh.
Large, long-term ADM contracts may include
AMS in application outsourcing deals
comprising Agile development, application
modernization and quality assurance services.
This quadrant specifically focuses on the AMS
services offered by providers.

Typical service levels include the time taken to
resolve an incident or service request, service
availability, the defect rate, user satisfaction or
Net Promoter Score (NPS) and user experience.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

and SQL Server); mainframeand other technologies can add
to a provider’s rating but are notrequired for inclusion

Eligibility Criteria

1.

Deployment and operationof service platforms for
performance and defectmanagement, including
troubleshooting, application
tickets and service requests

2.

Employment of vendor-certified
experts in packaged e-commerce,
ERP or CRM (at least one of thesecommercial applications)3.

Clearly supports Microsoft
and Oracle technologies, Java
programming and relational
databases (such as MySQL,
Oracle Database, PostgreSQL

4. Integration of more than two
service platforms, such as
Atlassian Jira, SAP Solution
Manager, ServiceNow andapplication platforms, such
as AWS, Google Anthos, IBM
Rational and Microsoft Azure

5.

Contracts are based on fixed
service fees or outcomes,
providing clients with options;staff augmentation is anaccepted exception

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

29

Application Managed Services

Observations
Enterprises strategically aim to optimize costand operational efficiency while managing theirapplication portfolios within the IT landscape.
Service providers have devised methodologies
that harness cutting-edge technologies suchas AI, automation and advanced analytics
to deliver quantifiable advantages to theirclientele. Leveraging data-driven approaches,
they prioritize enhanced UX while meticulously
adhering to predefined KPIs. The surging
cloud infrastructure adoption underscores thecriticality of proficiently managing applications
in such environments. Timely optimizationof infrastructure availability and sustained
application performance emerge as nonnegotiable imperatives. In response, service
providers have effectively aligned with market
expectations, harnessing the power of AI — toa certain extent, generative AI — to provide
comprehensive application managed services
for clients. AI’s predictive analytics capabilities
empower the proactive identification ofpotential performance fluctuations andvulnerabilities. This proactive stance facilitates

pre-emptive measures, ensuring continuousand seamless application operation. Service
providers focus on delivering outcome-based
pricing models based on the agreed-upon KPIs
and targets, enabling a mutually beneficial
situation for the provider and the client. Thissituation is expected to continue in addition
to the other application managed services
contract models.
From the 86 companies assessed for this study,
29 qualified for this quadrant, with 11 being
Leaders and one a Rising Star.

Accenture is known for its KPIs andbusiness value-driven approach in the ADM.
Its myWizard® platform uses structured
methodologies for automation. AI tools, alongwith more than 350 intellectual properties
and other assets, bring speed, efficiency,
agility and quality.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Capgemini drives cost reduction and businessoutcomes with vertical stack models anduses its eAPM and EAF for automation.
Implementing business XLAs empowers clientsto actualize business value and enhance CX.

Cognizant invests in innovative IPs such as
Cognizant Neuro® IT Operations, Intelligent
Virtual Agent and iGenie. These intellectual
properties modernize apps with the helpof automation, AI and analytics, focusing
on continuous improvement andoperational efficiency.

Deloitte’s robust service portfolio harnesses
AI, ML and a rich partner ecosystem to help
clients innovate, grow and manage risks in theircore business.

DXC’s Platform X™ enables data-driven
intelligent automation for resilient, self-healing
IT. It offers intelligent automation solutions
such as ASA and SPARK. Skilled resources and
IPs ensure incident detection and resolution
and can predict and prevent future problems.

HCLTech’s managed application services
focus on CX and CloudOps for secure app
management. NextGen ASM Framework
2.0 powered by iONA enables predictive
operations via extreme automation andAI-driven decision-making.

Infosys offers LEAP, an AI-driven analytics
platform, which integrates existing APM
and AIOps tools and uses ML algorithms forproactive monitoring, anomaly detection
and predictive maintenance. It also offers
accelerators, assets and chatbots to increase
AI adoption.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

30

Application Managed Services

Quinnox
LTIMindtree’s managed services prioritize
transformative capabilities and IT
transformation through a consulting-led
approach. JORITZ digitizes IT operations with
AIOps, and BRAIO streamlines end-to-end
processes with workflow automation, driving
enterprise-wide agility.

TCS MasterCraft™ optimizes softwaredevelopment and service delivery. Supported
by 24x7 services, it enables continuous
deployment, aligning business strategy andexecution. AI-based ignio™ automates alerts
and incident resolution and enhances visibility.

Tech Mahindra offers amplifAI0->∞ is an AI
and automation platform that supports digitaltransformation and offers anomaly detection,
auto resolutions, virtual assistance andincident analytics.

Quinnox’s (Rising Star) AMS enhances
IT landscapes via business-centered methods,
employing in-house tools and data-driven
evaluations. As a Rising Star, it has developed
IPs, including Qinfinite for asset discovery,
BizOps for monitoring and Qyrus for predictive
test automation.

Wipro employs smart operations, automation
and user-centric experiences to enhance
efficiency and resilience. Its platform-led
enablement leverages the HOLMES AI
platform and partner ecosystem for rapid
value realization.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

31

Application Managed Services
LeaderCognizant“Cognizant’s investments in Cognizant Neuro® andAppLens help the company focus on modernization
and automation using AI and analytics. Its AI-driven
operations emphasize automation, debt management
and SRE transformation for agility and reliability. ”
Akhila Harinarayan

Overview

Strengths

Cognizant is headquartered in New
Jersey, U.S. and operates in 42 countries.
It has more than 351,500 employees
across 162 global offices. In FY22 thecompany generated $19.4 billion inrevenue, with Financial Services as its
largest segment. In 2022, AMS services
in the U.S. contributed $2.2 billion to
the company’s revenue. The provider
offers a wide range of services such as
platform-led application management,
application debt and automation
management as-a-service, and SRE
transformation after finding the proper fit
for purpose and use. It leverages a global
network of studios, CoEs and innovation
labs to cater to clients across verticals.

Application modernization and automation:
Cognizant focuses on modernizing
and managing applications to meetbusiness needs effectively. The company
uses automation, AI and analytics todeliver scalable and reliable solutions.
Its AMS strategy emphasizes continuous
improvement and operational efficiency.
AI-driven operations: Cognizant’s application
maintenance practice, part of the SPE unit,
engineers modern applications for agile
businesses. It adopts an automation-first
approach, utilizing the Cognizant Neuro®
IT Operations platform to optimize
performance and reduce costs.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Investments in IPs and solutions:
Cognizant invests in innovative IPs and otherassets, including the Cognizant Neuro®
IT Operations platform with AI-powered
automation for improved resilience andvisibility. Others include the Intelligent Virtual
Agent, which offers precise, self-service
support, Cognizant AppLens, which drives
business relevance using a data-driven
approach, Cognizant Business Outcomesand Experience Dashboard, which enablereal-time monitoring, and iGenie, which
is an intelligent data analysis tool withvisualization capabilities.

Caution
The task automation level throughout
Cognizant’s managed application services
cycle is almost equal to the industry average.
In contrast, incidence elimination at
25 percent is slightly lower than the industry
average (30-35 percent). Cognizant should
focus on increasing the incidence eliminationlevels in its engagements.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

32

avv

Application Quality AssuranceApplication Quality Assurance

Who Should Read This Section
This report is relevant to enterprises acrossindustries in the U.S. for evaluating quality
assurance services providers.
Enterprises anticipate QA providers to ensure
application excellence through automated,
intelligent and continuous testing within Agileand DevOps contexts. Seamless security
integration (DevSecOps) is prioritized forcollaboration, security testing and threat
management. Requirements for this integrationencompass real-time vulnerability detection, AI
tools, app resilience and risk reduction.

Providers combine Agile, automation, AI,
DevOps and SRE principles for high app quality
and reduced maintenance. A robust partner
ecosystem caters to client needs. The quality
engineering (QE) approach enhances IT
app sustainability. AI-driven compliance
ensures real-time regulatory alignment.
Consulting follows an outcome-based approach
spanning quality, testing, DevOps and Agile.
Robust automation boosts testing efficiency,
and AI and ML in SDLC tackle app quality.

Providers must leverage advanced technologies
and proactive quality assurance to alignsolutions with business needs. Expedited
software release cycles across ADM require
tailored testing and DevOps integration.
Providers adapt QA practices, emphasizing
cloud-based automation with AI and ML.
Shift-left testing strategies, bolstered by
AI and ML automation, elevate app quality.
Site reliability engineering (SRE) principles
enhance app quality prior to deployment,
minimizing downtimes and optimizing UX.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

IT professionals should read this report
to determine service providers’ strengths
and weaknesses in ADM and learn how
to integrate cutting-edge technologies formarket advantage.

Product professionals should read thisreport to understand the market landscapeand provider offerings, which can be used to
improve the production process.

Business professionals should review
this report to understand partner
positioning for efficient application service
procurement and favorable ROI in theirbusiness or industry.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

34

Source: ISG RESEARCH

Next-Gen ADM Services
Application Quality Assurance

U.S. 2023

H

Accenture

PORTFOLIO ATTRACTIVENESS

HCLTechL

LTIMindtree
Kyndryl
DXC Technology
Product Challenger

Fujitsu
EncoraQuinnoxContender

Deloitte

Capgemini
Cognizant
Wipro
Infosys
TCS
LeaderQA Consultants
Infinite

Aspire Systems

Market Challenger

IBMIndiumInnominds

Mphasisa1qa
Trigent Software

L

This quadrant assesses
providers that offer
comprehensive
quality assurance (QA)
programs, including
assessments, design,
implementation and
QA services, such as
training and education
offerings for developers,
testers and operators.
Akhila Harinarayan

SLK Software

COMPETITIVE STRENGTH© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

H

Rising Star

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

35

Application Quality Assurance

Definition
This quadrant assesses service providers thatoffer comprehensive quality assurance (QA)
programs, including assessments, design,
implementation and quality assurance
managed services. Service deliverables include
setting methods for effort estimation, projectplanning, documentation, sprint execution
timelines, criteria for a product to be deemedcomplete, and testing strategies to identify
bugs or defects in a product.
Service providers in this quadrant can design
processes to attain the desired product orservice quality at project and business levels,
ideally covering a client’s complete application
portfolio. They leverage quality frameworks tosupport application code quality improvements,
infrastructure resiliency, digital testing, security
and quality assurance artifacts, and products
and vendor tools.

The quality assurance service should include
training and education to help clients mature
their software engineering capabilities.
A quality assurance program should involve all
the development teams, including experts fromthe outsourcing companies working for clients.
This quadrant also assesses how a provider
leverages production logs to extract insights for
improved application quality and performanceand how the provider integrates application
performance management tools with AI and
ML for data monitoring to predict the quality of
new applications.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Eligibility Criteria

1.

Centralized QA unit that lays
down quality standards forclients’ projects

2.

Comprehensive technical
QA framework, which includes
planning, implementation,
monitoring, review andimprovements

3.

Consulting team focused on
analyzing business demands
and securing development anddelivery according to the specificbusiness requirements

5.

Differentiation with proprietary
tools and multiple vendor
partnerships for quality
monitoring, application
performance tools andtesting tools

6.

Training and education offering
for developers, testers andoperators to develop a quality
excellence mindset and ensurethat the overall product or service
meets the desired quality

4. Applying technology toperform analytics overlogs and AI for continuous
improvement in results

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

36

Application Quality Assurance

Observations
Service providers are constantly challenged
to reduce software release cycles acrossADM. Adopting tailored testing solutions andintegrating DevOps practices and tools help
firms achieve this. Providers’ QA practices
are strategically tailored to prioritize delivery
experiences aligned with clients’ requirements.
They channel their efforts into crafting cloudbased automation tools and accelerators thatharness the potential of AI and ML to address
the escalating demand for swifter cycles.
SaaS-based testing tools employ codeless,self-healing and predictive test automation
mechanisms, culminating in an expedited andstreamlined software delivery pipeline.
Increasing demand for shift-left testing
strategies, fueled by AI- and ML-infused test
automation, has tangibly elevated application
quality. By performing comprehensive testing
earlier in the software development lifecycle,
enterprises proactively identify and rectify
potential anomalies, resulting in softwarewith increased resilience and dependability.

SRE demand has helped improve application
quality even before its deployment into
the production cycle. Through meticulous
orchestration of software assessment acrossvarious facets, SRE effectively reduces
application downtime, optimizing reliability
and UX.
From the 86 companies assessed for this study,
24 qualified for this quadrant, with eight being
Leaders and one a Rising Star.

Accenture combines Agile practices,automation, AI, DevOps and SRE principles
to help clients maximize application quality
and minimize maintenance requirements andcosts. Its robust partner ecosystem helps thecompany address personalized client demands.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Capgemini offers an accelerator that wasdeveloped based on a three-dimensional
quality engineering (QE) approach. It measuresand improves the sustainability of clients’ ITapplications. Its sustainable QE dashboard aids
in green assessment-driven reporting.

Cognizant’s AI-driven platform ensuresreal-time regulatory compliance for IT systems
and products under life science regulations.
It employs automation and AI technologies
for continuous software delivery andmaintaining compliance.

Deloitte’s Automation First Quality Engineering
Practice offers a comprehensive service
portfolio. Its rich array of IPs and assets drive
cutting-edge testing solutions. Its strategic
alliances with top testing tool vendors empower
it to provide unmatched QE services.

HCLTech employs an outcome-based approachwith its consulting team working in close
collaboration to offer end-to-end consulting
services for quality, testing, DevOps andAgile transformation.

Infosys uses curated prompts and fine-tuned
models for various use cases, including user
story to test case generation, user story toscript generation, script generation fromcommercial tools to open source and test
scenario generation from legacy code base.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

37

Application Quality Assurance

TCS specializes in QE processes,
methodologies and industry best practices.
Its well-developed testing frameworks
help in identifying defects and improving
product quality.

LTIMindtree (Rising Star) integrates AI and ML
into SDLC management to address application
quality challenges. The provider collects data
across the SDLC, mainly from user stories, test
cases, defects, code repository and APM logs,
among other sources.

Wipro uses robust test automation frameworks
to accelerate testing processes and improve
test coverage. It employs various tools andframeworks to create efficient and reusable
automation scripts.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

38

Application Quality Assurance
LeaderCognizant“Cognizant has a robust application quality
assurance practice. The provider has more than
30 proven AI use cases in addition to IPs andaccelerators across the testing lifecycle.”
Akhila Harinarayan

Overview

Strengths

Cognizant is headquartered in New
Jersey, U.S. and operates in 42 countries.
It has more than 351,500 employees
across 162 global offices. In FY22 thecompany generated $19.4 billion inrevenue, with Financial Services as
its largest segment. Cognizant has
a dedicated Quality Engineering &Assurance Practice (QE&A) to engineer
and deliver QE solutions for clients.
The QE&A practice is housed within
Cognizant’s Software & Platform
Engineering service line that focuses ontransforming experience with modern
engineering and quality.

AI-driven platform for compliance:
Cognizant offers an AI-driven platform todeliver real-time regulatory compliance forIT systems and products falling under life
science regulations. The platform enables
continuous software delivery with ongoingcompliance using automation and AI
technologies, which automate the validationand compliance assurance processes.
It has an in-depth understanding of providing
assurance for local compliances, such as
GxP compliance, HIPAA compliance, FDA
and FATCA, across industries.
Business-focused application QA: QA has
moved from assuring IT systems to protecting
the overall business and brand. Cognizant
has scaled Continuous Testing - Automation
First - QE approach to enterprise-level© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

DevOps with a platform-based lifecycle
quality orchestration. It offers innovative
QE solutions for applications and platforms
built on next-gen technologies, including
SaaS (S/4 HANA, Salesforce, Workday,
Guidewire), cloud hyperscalers, IoT, AR/
VR and AI, and for regulatory compliancewith BASEL 2, GxP, FDA, GDPR, MDR andISO 20022 migration.

CautionCognizant should focus on marketingand highlighting the benefits delivered toclients through outcome-based pricing for
application quality assurance. Currently,
Cognizant has low visibility in this area.

Powerful IPs: Cognizant has adopted aplatform-based approach to application
development, modernization, management
and QA. For example, its AppLens
management platform provides automationand technical debt reduction as on-demand
services and supports SRE functions.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

39

Continuous Testing Specialists

Continuous Testing Specialists

Who Should Read This Section
This report is relevant to U.S. enterprises forassessing continuous testing service providers.
Enterprises’ demand for continuous testing
services is on the rise as businesses acrossvarious industries in the U.S. seek to expedite
software and application development
cycles. Shorter time-to-market and increased
operational efficiencies are driving factors,
with testing integration into development
processes further reducing development timeand costs. The adoption of Agile methodologies
and DevOps/DevSecOps practices is also
contributing to the growing demand for testing
services. The cloud application surge leads to aquest for scalability, flexibility and cost-efficiency
through cloud-based testing and ADM services,
enabling real-world scenario replication andcomprehensive testing.

Providers specializing in continuous testing
must possess traits that prioritize QE, utilizing
AI and ML for efficient automation. They should
employ frameworks, accelerators and other IP
to automate test script creation, integrating AI
and ML. Providers’ proficiency should extend totheir proprietary platforms and tools, enablingtransformative and tool-agnostic automatedsolutions. Their team should be proficientin operating various automation tools andtesting frameworks, thus ensuring effective
service delivery.

IT professionals should read this report
to determine service providers’ strengths
and weaknesses in ADM and learn how
to integrate cutting-edge technologies
for market advantage.

Product professionals should read thisreport to understand the market landscapeand provider offerings, which can be used to
improve the production process.

Business professionals should review
this report to understand partner
positioning for efficient application service
procurement and favorable ROI in theirbusiness or industry.

Providers harness AI and automation to predict
defects, optimize test cases and enhance testing
efficiency. They also prioritize security testing
and compliance against cyber threats, andemphasize user-centric testing for enhanced
usability, performance, accessibility andresponsiveness across devices and platforms.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

41

Source: ISG RESEARCH

Next-Gen ADM Services
Continuous Testing Specialists

U.S. 2023

H

PORTFOLIO ATTRACTIVENESS

Qualitest
ZensarCybagePersistent Systems

Product Challenger

ITC Infotech

EvidenCigniti

UST

Tech Mahindra
HexawareLeaderApexonHTC Global
Contender

Birlasoft

Softtek

Coforge
Market Challenger

TestingXperts
N-iX

Marlabs

This quadrant evaluatesproviders of continuous
testing services
involving automatedsoftware updates testing
post-implementation
to prevent newfeatures and code
alterations from
causing regressions
or disruptions to theexisting functionality.
Akhila Harinarayan

Concentrix

L

L

COMPETITIVE STRENGTH© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

H

Rising Star

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

42

Continuous Testing Specialists

Definition
This quadrant assesses providers of continuous
testing services. Continuous testing is theprocess of automatically testing softwarechanges as they are made to ensure that newfeatures and code changes do not introduce
regressions or break existing functionality.
Providers execute application testing, including
defining the testing strategy, scope, methods
and scripts. They also can differentiate thebest approach to manual testing before
consuming automation resources in test
execution. These providers have the skills to
deploy automation, execute testing cycles andproduce the necessary evidence to supportcompliance auditing.

This discipline goes beyond automation-based
testing regarding people and processes; it
accomplishes better collaboration between
the quality assurance and development teamsin sprint cycles, besides feature-driven testing
and responsiveness to changes. Providers’portfolios may include unit testing, system
testing, regression testing, compliancetesting, performance and load testing, user
acceptance testing and smoke testing.
The most comprehensive portfolio gets a
better appraisal.

Continuous application testing focuses ondelivering quality in tandem with the speed ofAgile development. In terms of technology, it
encompasses various aspects of automatedtesting, such as shift-left and end-to-end
automation across testing phases, in every
phase of the continuous delivery process.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

4. Continuous services, including

Eligibility Criteria

1.

Qualified professionals fortest-driven development (TDD),
behavior-driven development(BDD) and other approaches

2.

Capability to handle
large-scale testing andcontinuous integration demandsof complex systems, such as
ERP and e-commerce withmany test cases

3.

testing data and test coverageassessments, enabling
automated testing across many
continuous integration pipelines,
and managing testing artifacts
for the significant reutilizationof such artifacts

5.

Replication and reuse of
testing artifacts to use inmultiple projects

Consulting services that include
test automation implementation,
which can be integrated withclients’ development and
DevOps tools and help clients
optimize their continuous
testing performance to reduce
the testing time

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

43

Continuous Testing Specialists

Observations
The surge in cloud application adoption
has driven enterprises to seek enhanced
scalability, flexibility and cost-efficiency
through cloud-based testing and ADM services.
This scenario facilitates the replication ofreal-world scenarios and comprehensive
testing across diverse platforms. The need todeliver efficient applications under mountingpressure has led service providers to use AI
and automation. These technologies empower
providers to assist clients in identifying
patterns, predicting potential defects andoptimizing test cases. AI-driven testing andautomation not only enhance testing efficiency
but also proactively improve application quality
by predicting potential issues.

Enterprises are pivoting toward user-centric
testing that transcends traditional functional
assessment. This approach entails evaluating
an application’s usability, performance,
accessibility, responsiveness and compatibility
across a spectrum of devices and platforms.
Overall, the interplay of cloud-based testing,AI-driven approaches, security testing andUX evaluation are the main trends for thecontinuous testing specialist domain. By
integrating these elements, enterprises
and service providers collectively focus onheightened application quality, robust security
and improved UX.
From the 86 companies assessed for this study,
19 qualified for this quadrant, with eight being
Leaders.

The escalating frequency of cyber threats
and data breaches compels firms to prioritize
security testing and compliance, leading
to the development of robust security
practices that encompass vulnerability
assessments, penetration testing andcode analysis to safeguard applications.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

ApexonApexon offers testing services via its
QA portfolio, which focuses on systems,
processes and automation to enhance quality,
agility and efficiency throughout every stage ofthe digital application development lifecycle.

Birlasoft’s testing services prioritize QE andutilize AI and ML for streamlined automation,enhancing value by reinforcing QA in
applications and business processes.
It deploys frameworks, accelerators and other
IPs to automate test script generation.

Eviden is an expert in the testing domain
with its exclusive platforms and tools,
providing transformative and tool-agnostic
automated services. Its staff members aretrained in diverse automation tools andtesting frameworks, facilitating efficient
service delivery.

Hexaware’s ATOP, powered by AI and ML, is
a comprehensive testing platform coveringfunctional and nonfunctional testing acrossall layers. It automates multichannel testing,accelerates big data and ETL testing, andprovides service virtualization.

Cigniti provides continuous testing services,integrating QA into the delivery lifecycle.
It implements shift-left and shift-right testingprocedures, enabling early defect detection.
Its proprietary QE platform, BlueSwan™,
incorporates AI and ML for test automation.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

44

Continuous Testing Specialists

Qualitest
Qualitest’s next-gen QE, customer-centric GTM
approach and strategic acquisitions drive its
leadership in advanced testing. With AI, DevOps
and AR/VR/MR, the company redefines quality
assurance, serving diverse clients throughtailored solutions and inorganic growth.

UST offers NoSkript™, which enhancesengineers’ skills, reduces rework, accelerates
testing and automates the test lifecycle.
Past UST implementations show that thecompany has skilled staff and capabilities.

Tech Mahindra offers testing solutions
such as MAGiX, an automation framework,
eConvergence for integrated reportingand eAnalytics for predictive analytics.
It possesses expertise across various testing
frameworks, driven by people skilled in multiple
automation tools.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

45

Appendix

Methodology & TeamThe ISG Provider Lens™ 2023 – Next-Gen
ADM Services study analyzes the relevant
software vendors/service providers in theBrazilian, European and U.S. market, based ona multi-phased research and analysis process,and positions these providers based on the
ISG Research™ methodology.

Information Services Group Inc. is solely
responsible for the content of this report.
Unless otherwise cited, all content, including
illustrations, research, conclusions, assertions
and positions contained in this report weredeveloped by, and are the sole property ofInformation Services Group Inc.

Lead Author:
Akhila Harinarayan

The research and analysis presented in thisreport includes research from the ISG Provider
Lens™ program, ongoing ISG Research™
programs, interviews with ISG advisors,
briefings with services providers and analysis
of publicly available market information frommultiple sources. The data collected forthis report represents information that ISG
believes to be current as of August 2023, forproviders who actively participated as well as
for providers who did not. ISG recognizes thatmany mergers and acquisitions have takenplace since that time, but those changes are
not reflected in this report.

Editors:
Poulomi Nag and John Burnell
Research Analyst:
Maharshi Pandya
Data Analysts:
Rajesh MC and Anusha R
Consultant Advisors:
Bill Shoemaker and Jerry Lawson
Project Manager:
Abhishek Rammurthy

The study was divided into thefollowing steps:

1. Definition of Next-Gen
ADM Services market

2. Use of questionnaire-based

surveys of service providers/
vendor across all trend topics

3. Interactive discussions withservice providers/vendors on
capabilities & use cases

4. Leverage ISG’s internal databases& advisor knowledge & experience(wherever applicable)

5. Use of Star of Excellence
CX-Data

6. Detailed analysis & evaluation ofservices & service documentation
based on the facts & figures
received from providers & othersources.

7. Use of the following key
evaluation criteria:* Strategy & vision* Tech Innovation* Brand awareness andpresence in the market* Sales and partner landscape* Breadth and depth of
portfolio of services offered* CX and Recommendation

All revenue references are in U.S. dollars ($US)
unless noted.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

47

Author & Editor Biographies

AuthorEnterprise Context and Global Overview Analyst

Akhila Harinarayan

Maharshi Pandya

Lead Analyst

Research Analyst

Akhila Harinarayan is Senior Lead Analyst
and the lead author for ISG Provider Lens
studies with a focus on Digital BusinessTransformation and SAP Services.
She has more than 12 years of experience
across research and consulting including
provider strategy, enterprise strategy,
industry roadmaps, point-of-view papers,
service provider assessment across regions.
She has strong expertise on strategy andtransformation, digital insights, thought
leadership, benchmarking, market
assessments and go-to-market strategies.

She has authored many thought leadership
papers, digital insight studies, devised
go-to-market strategies across products/
industries/regions, built frameworks andmaturity models across industries for both
enterprises, vendors and service providers.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

Maharshi Pandya is a Research Specialist at
ISG and is responsible for supporting andco-authoring ISG Provider Lens™ studies
on SAP HANA Ecosystem & Next-Gen
ADM Solution and Services. He supports
the lead analysts in the research process
and authors the global summary report.
Maharshi also develops content from anenterprise perspective and collaborates withadvisors and enterprise clients on ad-hoc
research assignments as well. Prior to thisrole, he has been associated with several
syndicated and custom market researchfirms, in which he has worked on both,secondary and primary interaction centricresearch projects around market sizing &forecasting, competitive benchmarking,pricing analysis vendor profiles and market
share analysis for several industry verticals
such as information and communication
technology, media & information services,
and automotive. His area of expertise
includes analytics, application development
and maintenance, and enterprise
resource planning.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

48

Author & Editor Biographies

IPL Product Owner

Jan Erik Aase
Partner and Global Head – ISG Provider Lens™
Mr. Aase brings extensive experiencein the implementation and research ofservice integration and management ofboth IT and business processes. With over
35 years of experience, he is highly skilled
at analyzing vendor governance trends andmethodologies, identifying inefficiencies incurrent processes, and advising the industry.
Jan Erik has experience on all four sidesof the sourcing and vendor governancelifecycle - as a client, an industry analyst,
a service provider and an advisor.

Now as a research director, principal analyst
and global head of ISG Provider Lens™,
he is very well positioned to assess andreport on the state of the industry and makerecommendations for both enterprises andservice provider clients.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

49

AboutOur Company
Methodology& Team& Research

The ISG Provider Lens™ Quadrant researchseries is the only service provider evaluationof its kind to combine empirical, data-driven
research and market analysis with thereal-world experience and observationsof ISG’s global advisory team. Enterprises
will find a wealth of detailed data andmarket analysis to help guide their selectionof appropriate sourcing partners, while
ISG advisors use the reports to validate
their own market knowledge and makerecommendations to ISG’s enterpriseclients. The research currently covers
providers offering their services acrossmultiple geographies globally.
For more information about
ISG Provider Lens™ research,please visit this webpage.

ISG Research™ provides subscription
research, advisory consulting andexecutive event services focusedon market trends and disruptive
technologies driving change in businesscomputing. ISG Research™ delivers
guidance that helps businesses
accelerate growth and create
more value.
ISG offers research specifically
about providers to state and local
governments (including counties, cities)
as well as higher education institutions.
Visit: Public Sector.
For more information about ISG
Research™ subscriptions, pleaseemail contact@isg-one.com,
call +1.203.454.3900, or visit
research.isg-one.com.

© 2023 INFORMATION SERVICES GROUP, INC. ALL RIGHTS RESERVED.

ISG (Information Services Group)(Nasdaq: III) is a leading global technology
research and advisory firm. A trusted
business partner to more than 900 clients,
including more than 75 of the world’s
top 100 enterprises, ISG is committed
to helping corporations, public sector
organizations, and service and technology
providers achieve operational excellence
and faster growth. The firm specializes indigital transformation services, including
automation, cloud and data analytics;
sourcing advisory; managed governanceand risk services; network carrier services;
strategy and operations design; change
management; market intelligence andtechnology research and analysis.

Founded in 2006, and based in Stamford,
Conn., ISG employs more than 1,600 digitalready professionals operating in more than
20 countries—a global team known for
its innovative thinking, market influence,
deep industry and technology expertise,
and world-class research and analyticalcapabilities based on the industry’s mostcomprehensive marketplace data.
For more information, visit isg-one.com.

NEXT-GEN ADM SERVICES REPORTSEPTEMBER 2023

50

SEPTEMBER, 2023
REPORT: NEXT-GEN ADM SERVICES© 2023 Information Services Group, Inc.
All Rights Reserved