IBM watsonx

A differentiatedapproach to AI
foundation models
Scale generative AI with enterprise-grade models

Table of contents
03

06

Introduction

IBM Granite, developed by
IBM Research: An example
of IBM’s differentiatedapproach to models

04
The needs of enterprises
as they scale generative AI

09

05

Conclusion: The outcomesof successfully scaling
and operationalizing
generative AI

IBM’s differentiatedapproach to delivering
enterprise-grade
foundation models

2Introduction
In 2023, organizational departments such as HR, IT andcustomer care have focused on the generative AI use cases,
such as summarization, code generation and Q&A to take out
costs and drive productivity. According to a recent study fromthe IBM Institute for Business Value (IBV), 75% of CEOs believe
competitive advantage will depend on who has the most advanced
generative AI and 50% are now integrating generative AI into
their products and services.1
The IBV study also deduces that the experimentation phasefor generative AI leaders will be short and intense, with 74%
of executives reporting that generative AI will be ready for
general rollout in the next three years. Multimodal, multilingual
foundation models and automation agents will expand thegamut of generative AI use cases and adoption across business
workflows. Additionally, vertical and domain-specific foundation
models with a smaller number of parameters that can match
the performance of larger models at lower cost of inferencing
are predicted to gain more market traction. Furthermore, thetechniques for training foundation models will continue to
evolve, unlocking advanced capabilities and new orders ofefficiencies to make generative AI even more attractive
for businesses.

61%of CEOs identify concerns
about data lineage andprovenance as a barrier
to adopting generative AI.

3

A mandate to embrace generative AI with eyes wide open
Business and technology executives are handed a clear mandate
from their leadership and boards to transform their businessmodels, offerings and operations with generative AI in 2024. An
IBM study on responsible AI and ethics finds that CEOs say they
feel over six times more pressure from their boards and investors
to accelerate the adoption of generative AI rather than slow
it down.2
According to the IBM CEO study, 58% of business executives
believe that major ethical risks abound with the adoption ofgenerative AI, and 79% say AI ethics is important to theirenterprise-wide AI approach.2 61% of CEOs identify concerns
about data lineage and provenance as a barrier to adopting
generative AI,1 even as 85% of executives anticipate directinteractions between generative AI and customers within thenext two years.1
Policymakers and regulators are waking up to the potential risks
and social harms of generative AI and are drafting policies andlaws to ensure sustainable innovation and diffusion. Both the EU
AI Act and the US White House executive order on AI announced
in 2023 signal government commitment to the scrutiny of AI.
In addition, business leaders are concerned about exposing
their company’s proprietary data to foundation models hosted
in the cloud, yet they see the value in fine-tuning AI with its
business context.
Taking steps to ensure the value exceeds the cost
Cost is another major consideration of policymakers andregulators. According to an article in the scientific journal Nature,a search driven by generative AI uses 4–5 times the energy
needed to run a conventional web search, and large models inmarket require volumes of expensive GPU compute resources.3
Enterprise decision-makers leading generative AI initiatives
within their organizations have a lot to think about. As
organizations move from exploration to investigation andproduction with generative AI, they need foundation models
to power key tasks in various departments. In this white paper,
we share the value of enterprise-grade foundation models thatprovide trust, performance and cost-effective benefits toall industries.

A differentiated approach to AI foundation models

The needs of enterprises
as they scale generative AI
The major challenge facing enterprise decision-makers is striking
a balance between scaling generative AI faster across a globalorganization that leverages its proprietary data and minimizing
foundational model-related risk exposure and total cost ofownership (TCO).
Moving from exploration to investigation and production
with generative AI will require enterprises to adopt the rightmodel choices for the right use cases and a robust platform
to customize models and infuse AI into their applications.
In addition, they’ll need hybrid cloud to deploy AI in theirinfrastructure of choice and a reliable partner who can help
scale and operationalize AI with minimal risks.
A strategic approach is key to successful adoption
As decision-makers evaluate options to choose the rightfoundation models for scaling generative AI, it’s crucial toconsider a strategic point of view that incorporates both
enterprise-grade foundation models and a robust platform
to operationalize them.

Customization, specialization and flexibility
Users should be able to customize the foundation models fortheir use cases, company and industry domain using fine-tuning
and prompt-tuning with an easy-to-use toolkit. They would alsoneed specialized database capabilities for storing, managing
and retrieving high-dimensional vectors that fuel generative
AI applications. Depending on the use case, the nature of
the content used to inference the model and operational
considerations, clients should have the flexibility to deploy themodel in the infrastructure of their choice. AI guardrails andcontinuous monitoring make model deployments secure andreliable as organizations scale-up generative AI applications.
Enterprise decision-makers also seek a reliable technology
partner that grasps opportunities and risks in enterprise AI
adoption, understands the key model dimensions and bakesin AI ethics and regulatory preparedness across the generative
AI lifecycle, starting with foundation model development.

Trust in foundation models, gauged throughmetrics like transparency indexes andhallucination scores, hinges on transparency
in data management, training andevaluation processes.

Performance measures can then be used todetermine which attributes, such as versatility,
accuracy, latency and throughput, are critical
for the enterprise use case.

Cost-effectiveness measures can help narrow
down model choices that deliver the necessary
performance at lower inferencing costs andwith fewer compute resources.

4

A differentiated approach to AI foundation models

IBM’s differentiated approachto delivering enterprise-grade
foundation models

You can’t scale generative AI
with models that you can’t trust
TrustedEnterprise-grade foundation models represent a class of models
characterized by their trustworthiness, performance and costeffectiveness, which makes them well-suited for deployment
in enterprise settings where reliability, efficiency and value areparamount considerations. While most model providers obsess
about building and bringing the most capable models to themarket, enterprise clients should step back and asses what
optimal mix of model attributes they need to succeed.
IBM’s approach to delivering enterprise-grade models
encompasses four key principles:
1. Open: Bring best-in-class open models from IBM and open
source to our IBM watsonx™ foundation models library.
2. Trusted: Train models on trusted and governed data for
applications that require enterprise-level transparency,
governance and performance in accordance with IBM AI
Ethics principles.
3. Targeted: Design for the enterprise and optimize fortargeted business domains and use cases with opensourced or insourced skills and knowledge, using largescale alignment techniques.
4. Empowering: Provide clients with competitively priced
model choices to build AI that best suits their uniquebusiness needs and risk profiles.
Open models and third-party models offer choice
IBM pursues a multimodel strategy with enterprise-grade
multimodal (text, code audio, image, geospatial) and multilingual
model choices that best suits clients’ unique business needs,
regional interests and risk profiles. In adherence to its opensource and ecosystem strategy, IBM gives clients the freedom
to choose from a diverse library that includes proprietary,open-source models and third-party models hosted on IBM®
watsonx.ai™, our next-generation enterprise studio for
AI builders.

OptimalSkewedPerformant
Figure 1. IBM’s point of view on enterprise-grade foundation
models—trusted, performant, cost-effective

Key benefits of the watsonx AI and data platform– Model customization and choice: Tailor models with
proprietary data and expertise to unique use cases andcompany and industry domains for easy integrations
with ready-to-use AI applications. The watsonx platform
brings complementary capabilities, such as vector databaseand embedding models, to support popular use cases likeretrieval-augmented generation (RAG). With the Bring-YourOwn-Model (BYOM) capability, clients can enjoy much more
flexibility and freedom over their model choices on watsonx.
– Robust governance: Make AI safe and secure at scale with AI
guardrails, continuous risk monitoring, integrated governanceand third-party large language model (LLM) support.
– Flexible deployment: Work with your infrastructure of choicewith hybrid multicloud and on-prem options to avoid vendor
lock-in and reduce TCO.

Clients can harness the IBM foundation model library directlyon the watsonx.ai platform. IBM watsonx.ai is a next-generationenterprise studio. Part of the watsonx™ AI and data platform,
it brings together new generative AI capabilities powered byfoundation models and traditional machine learning (ML) intoa powerful studio spanning the AI lifecycle. It allows you totune and guide models with your enterprise data to meetyour needs with easy-to-use tools for building and refining
performant prompts.

5

Cost-effective

A differentiated approach to AI foundation models

IBM Granite, developed by
IBM Research: An example of IBM’s
differentiated approach to models
IBM Granite™ is IBM’s flagship series of enterprise-grade
foundation models based on decoder-only transformer
architecture. Granite models are designed to possess an optimal
mix of model attributes to meet the trusted, performant andcost-effective requirements of the enterprise. Granite language
models are trained on trusted enterprise data spanning internet,
academic, code, legal and finance data sources.
Currently, we have the following models in the Granite series:

IBM Granite was trained onenterprise-relevant content thatmeets rigorous data governance,
regulatory and risk criteria definedand enforced by IBM’s AI Ethics
principles and its Office of Privacy& Responsible Technology.

1. Granite-13b-chat-v2.1: A chat model optimized for
dialogue use cases that works well with virtual agentsand chat applications
2. Granite-13b-instruct-v2.1: An instruct model trained on highquality finance data to perform well in finance domain tasks
3. Granite-20b-multilingual: Trained to understand and generate
text in English, German, Spanish, French and Portuguese
4. Granite-8b-japanese: Designed to perform language tasks
on Japanese text
5. Granite-7b-lab: Supports general-purpose tasks and is tuned
using the IBM’s large-scale alignment of chatbots (LAB)
methodology to incorporate new skills
6. Granite code models: A family of models trained in 116
programming languages and ranging in size from 3 to 34
billion parameters—with base model and instruction-following
model variants
A foundation model with trust built inIBM’s approach to developing the Granite series of foundation
models is guided by our AI ethics policies, with a strong emphasison trust and transparency in training data and the model-training
process. IBM stands behind its models by offering clients
intellectual property (IP) indemnity protection so they can focus
on the business impact of AI, not on costly courtroom litigation.
Granite models were built for the enterprise and governed byrules and safeguards to minimize hateful and profane content,or “HAP.” The models are trained on datasets that meet IBM’s
rigorous data governance, risk and compliance criteria.
IBM helps ensure the integrity of its Granite models throughcomprehensive data management across the training phases
to build trusted models.

6

A differentiated approach to AI foundation models

Dataset preprocessing(model-agnostic)

Dataset acquisition

General language
datasets

Data preprocessing(model-specific)

Domain-specificdatasets

Data collectionand extracion
IBM Cloud®
Object Storage

Data
deduplicationData
annotation– Document ID generation– Deduplication– Sentence splitting– Language detection– Hate, abuse, profanity– Document quality– URL blocklisting– License detection (code)Datafiltering

Tokenization

IBM internal
datasets

Key data acquisition stakeholders– Domain experts– IBM Legal– Chief Privacy Office– Responsible business alignment (IBM Procurement)– IBM AI Ethics Board

Figure 2. The training phases of the Granitemodels used to build trusted models

Improved accuracy for targeted
enterprise business domains likefinance and use cases like RAG is
achieved through chat fine-tuning
and model alignment techniques.

Chat fine-tuning techniques were designed to prevent
hallucinations and misalignments in the model outputs.

7

A differentiated approach to AI foundation models

The Granite-13b-v2.1 foundation model demonstrates someof the least biased behavior of studied models, according
to a measures of bias in open-ended language generation
dataset (BOLD) evaluation. It also consistently scores highon harmlessness across attack domains, according to
AttaQ dataset evaluation.⁴

With the IBM Granite series, picka competitively priced model withless infrastructure requirement,
IP indemnification and an easy-touse toolkit for model customization
and application integration.

Granite models were built for theenterprise and governed by rules
and safeguards to minimize hatefuland profane content, or “HAP.”

A foundation model that’s performant
Bigger isn’t always better—smaller and more targeted models
can perform on par with larger general-purpose models.
– According to IBM internal benchmark testing in February
2024, the Granite-13b-chat-v2.1 model showed almost
comparable performance across several key enterprise use
cases—summarization and entity extraction—when comparedto llama-13b-chat.⁵– According to IBM internal benchmark testing in March 2024,
the Granite-20b-multilingual model approaches the qualityof three popular open-source models—llama-2-13b-chat,llama 2-70b-chat and mixtral 8x7b—across three languages—
German, Portuguese and Spanish—within translation tasks
when compared to llama-2-13b-chat.⁵– In just three months, the new Granite-13b-chat-v2.1 hasachieved a 45% improved performance in RAG comparedto the earlier v1 model, based on internal IBM performancebenchmark testing for enterprise tasks that include RAG.⁵– According to IBM internal benchmark testing in Feb 2024,
on ConFinQA, a standard benchmark measuring accuracyof multiturn numeric reasoning in financial services Q&A,
the Granite.13b.chat.v2.1 model demonstrated 18%
better performance than the second best model llama2.70.
chat model.⁵– According to IBM internal benchmark testing in 2024, inJCommonsenseQA, a multiple-choice question answering
dataset in Japanese that tests commonsense knowledge,
Granite-8b-japanese outperformed open-source elyza-7bllama-japanese by more than 2 times for zero-shot and 1.2
times for few shot.⁵– Evaluation on a comprehensive set of tasks has shownthat Granite code models consistently match state-ofthe-art performance among open-source code LLMs
currently available.⁵
With lower latencies, IBM Granite models have been shownto generate faster responses for summarization tasks.
A foundation model that’s cost-effective
Granite models require only a fraction of GPU capacity
and compute, which can result in a lower carbon footprint
and reduce TCO.
– Granite models are competitively priced in the market
and can run on less than 1 GPU.
– IBM Watson large speech model, which is 5 times smaller
than OpenAI’s Whisper, processes audio 10 times fasteron the same hardware, which can translate to lower costs.6

8

A differentiated approach to AI foundation models

Conclusion: The outcomes of successfully
scaling and operationalizing generative AI
As we reflect on the transformative impact of generative AI
and foundation models across industries, business domains
and geographies, it’s evident that the integration of generative
AI isn’t just imminent but already in motion. More than half oforganizations are actively exploring its capabilities. With theforthcoming expansion into multimodal and smaller specialty
models, businesses will further accelerate the impact of
AI. The journey ahead calls for executives to balance rapid
implementation with ethical and risk considerations, whilenavigating the fast-paced evolution of generative AI.
A leader in enterprise AI and hybrid cloud, IBM is well-positioned
in the marketplace to consistently bring trusted, performant
and cost-effective generative AI products and solutions to ourclients. IBM Research® will continue to evolve Granite models
through open-source engagement to deliver more performant
and trusted models at competitive pricing. Through the opensource project, InstructLab, initiated by IBM and Red Hat®,
we democratize model development and alignment with
open-sourced skills and knowledge.
Enterprises already leveraging IBM foundation models
on watsonx.ai– The Championships, Wimbledon used watsonx.ai foundation
models to train their AI to create tennis commentary andgenerate informative and engaging video clip narrations
for fans with varied sentence structures and vocabulary.7– Bradesco, a prominent financial institution in Latin America,
is excited about using watsonx.ai foundation models toenhance its AI strategy within its cutting-edge
corporate infrastructure.
– NASA is using foundation models to make it easier to build
AI applications for text and sensor data, enabling models
that detect natural hazards and track changes to vegetation
and wildlife habitat.
– Quantum Street AI is applying IBM foundation models
to provide investment managers with AI and ML tools
to identify market opportunities and manage risk.

9Take the next steps
Experience our interactive demo and sign up for a trialof watsonx.ai.
Additional resources:– Visit our website to learn more about IBM’s model PoV
and offerings.
– Learn more about IBM Granite models—data sources, training
steps and performance evaluations—by reading the latest
research paper.
– Use our model evaluation guide for simple decision heuristics
you can apply to refine your model choices.

A differentiated approach to AI foundation models

1. The CEO’s guide to generative AI: What you need toknow and do to win with transformative technology,
IBM Institute for Business Value, January 2024.
2. The CEO’s guide to generative AI: Responsible
AI & ethics, IBM Institute for Business Value,
23 October 2023.
3. Generative AI’s environmental costs are soaring—and mostly secret, Springer Nature Limited,
20 February 2024.
4. Watsonx foundation model evaluation summary, IBM
Research AI and watsonx.ai teams, 23 January 2023.
5. “Open sourcing IBM’s Granite code models,” IBM
Research blog, 6 May 2024.
6. IBM Research benchmarking study on customer careuse cases where each turn in the conversation lasted
less than 30 seconds.
7. IBM, Wimbledon and the power of watsonx, IBM Corp.,
January 2024.

© Copyright IBM Corporation 2024
IBM Corporation
New Orchard Road
Armonk, NY 10504
Produced in the
United States of America
May 2024
IBM, the IBM logo, IBM watsonx, watsonx, watsonx.ai, Granite, IBM Cloud, and
IBM Research are trademarks or registered trademarks of International Business
Machines Corporation, in the United States and/or other countries. Other productand service names might be trademarks of IBM or other companies. A current listof IBM trademarks is available on ibm.com/trademark.
Red Hat is a trademark or registered trademark of Red Hat, Inc. or its subsidiaries
in the United States and other countries.
This document is current as of the initial date of publication and may be changed
by IBM at any time. Not all offerings are available in every country in which
IBM operates.
All client examples cited or described are presented as illustrations of the mannerin which some clients have used IBM products and the results they may haveachieved. Actual environmental costs and performance characteristics will varydepending on individual client configurations and conditions. Generally expected
results cannot be provided as each client’s results will depend entirely on theclient’s systems and services ordered.
THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY
WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTYOR CONDITION OF NON-INFRINGEMENT.
IBM products are warranted according to the terms and conditions of theagreements under which they are provided.
The client is responsible for ensuring compliance with laws and regulations
applicable to it. IBM does not provide legal advice or represent or warrant thatits services or products will ensure that the client is in compliance with any law
or regulation.