The Past, Present, and Future of Generative AI:
Key Innovations and Trends
Exploring the Evolution and Impact of Generative AI
Technologies
APIGA 2024
Busan, July 10, 2024

Naufal Suryanto, Ph.D.
Postdoctoral Researcher @ IoT Research Center,Pusan National University
naufalso@pusan.ac.kr; naufal@islab.re.kr;
naufalsuryanto@gmail.com;
1

Disclaimer!
Welcome to our presentation, Dear APIGA Participants!
The content of this presentation was developed with the assistance of Generative AI
Technology.

Scan to access presentation material
https://bit.ly/apiga_genai
3

1. Introduction of Generative AI

2. History of Generative AI

3. Recent Advances in Generative AI

4. Applications of Generative AI

5. Ethical Considerations, Future Directions, and Conclusion
41. Introductionof Generative AI-5-1Introduction▪ What is Generative AI?

• Generative AI refers to a category of Artificial Intelligence (AI) algorithms designed to generate new data
instances that closely mimic the training data.

CATCAT

TrainedAI
ModelTrained
AI
Model
DOG

DOG

Discriminative AI

Generative AI
6

1Introduction▪ How Generative AI Works?

• Generative AI model is trained to learn the patterns and structures within a dataset. This allows it tocreate new data that looks similar to the original and follows the same underlying distribution.

Train
Train
DOG!

Predict
Predict

Generative

Discriminative
7

1Introduction▪ Generative AI Around Us

Text Generation

Video Generation

Image Generation

Auto Text Completion

Voice Assistant82. History ofGenerative AI-9-

2

History of Generative AI▪ The Early Beginning of AI

Alan Turing introduces theTuring Test to measure amachine's ability to exhibit
intelligent behavior.

Frank Rosenblatt develops
the Perceptron, the first
Artificial Neural Network.
Seppo Linnainmaa

1970

1955
1950

Alan TuringPaul Werbos

1982

1957

The term AI (ArtificialIntelligence) is coined forthe first time by John
MacCarthy.

Backpropagation algorithm
was formalized and applied
to allow neural networks tolearn from data.

Neural Network

Frank Rosenblatt
10

2

History of Generative AI▪ Deep Learning Era - The Key Breakthroughs of Generative AI*Unsupervised Learning*Revolutionize Generative AI*Revolutionize NLP
and Modern AI
Architecture

Autoencoders (AE)Generative Adv. Network (GAN)
2012

2015
2014

2000s

Transformers
2019
2017

ImageNet & AlexNet

Variational Autoencoders (VAE)

StyleGAN*Large Dataset & Model*Statistical Inference*Realistic & controllable

113. Recent Advances inGenerative AI-12-3

Recent Advances in Generative AI▪ Large Language Models (LLMs)

Large Text Datasets

Large Model

Large Compute Power

Trillions of texts (docs,articles, book, etc.)

Billions of neural network
params (Transformers)

Months of GPUs training

Large Language Model
AI models designed to understand, generate, and manipulate human language using deeplearning techniques

Foundational Model
Question Answering
Information Extraction

Summarizing

Text Generation

Sentiment Analysis
13Chatbot

Text Classification
Language Translation

Etc.

3Recent Advances in Generative AI▪ Large Language Models (LLMs)

How it works?
• Language Model is trained to predict the next
word given the text inputs.
Illustration:
The cat is so cute14

Trained Language Model

Repeatuntil <EOS>

cute [75%]fluffy[10%]playful[5%]…

Randomly sampled
based on output
probabilities

3

Recent Advances in Generative AI▪ Diffusion Models• Diffusion models are type of generative model that generate new data by starting with a noisy andgradually making it clearer and more detailed.

• Applications:15

3

Recent Advances in Generative AI▪ Demo (DALL·E 2 Explained) Text-to-Image Diffusion Model

16

4. Applications ofGenerative AI-17-4

Applications of Generative AI▪ Content Creations

Image Generation

Video Generation

Text Generation

Music Generation

Code Generation
18

4

Applications of Generative AI▪ Healthcare

Source:https://aicheckup.substack.com/p/wh
ere-generative-ai-meets-healthcare

19

4Applications of Generative AI▪ General Businesses

Workspace Applications

Enterprise chatbot

Benefits:• Efficiency: Automates repetitive tasks.
• Cost and Time Reduction: Optimizes processes, reduces costs and time.
• Innovation: Facilitates creative experimentation and new product development.
20

AI Meeting Assistant4Applications of Generative AI▪ Impacts (by Accenture Research)21

5. EthicalConsiderations, Future
Directions, andConclusion-22-5

Ethical Considerations, Future Directions, and Conclusion▪ Ethical Consideration

A faked video of the Ukrainianpresident telling people tosurrender is being shared online.

Bias and Fairness
Solution: Diverse and inclusive
training data with bias mitigation.

Privacy and Data Security
Solution: Prioritizing user privacy
and securing training data.

23

Misinfo. and DeepfakeIntellectualProperty,
Accountability,and Transparency

Solution: Detecting & preventing Solution: Robust regulation,
the misuse of generative AI.
compliance, and collaboration.

5

Ethical Considerations, Future Directions, and Conclusion▪ Future Directions

Image Source:
https://pub.towardsai.net/foundation-models-37074a2d70a1

Image Source:
https://medium.com/@henryhengluo/intro-of-ai-agent-ai-agent-proj
ects-summary-52f4a364ab86

Multi-Modal Fondation Models

AI Agents24

5

Ethical Considerations, Future Directions, and Conclusion▪ Conclusion
Recap of Generative AI Innovations●●Historical Progress: From early neural networks to modern breakthroughs like GANs and transformers, generative AI has evolved significantly.
Recent Advances: Highlighted the impact of large language models, diffusion models, and multimodal models on various industries.

Applications Across Sectors●●●

Business: Transforming content creation, customer service, marketing, and product design.
Healthcare: Accelerating drug discovery, enhancing medical imaging, and personalizing treatment plans.
Ethical Considerations: Addressing bias, privacy, misinformation, intellectual property, and accountability.

Future Directions●●Multimodal Foundation Models: Integrating multiple data types to enhance AI capabilities.
AI Agents: Autonomous systems performing complex tasks and making decisions.

Key Takeaways●●●Generative AI is transforming various industries by enabling more efficient and innovative solutions.
Ethical considerations are crucial to ensure responsible development and deployment of AI technologies.
Future advancements in AI will continue to shape our world, making collaboration and ongoing learning essential.

25

“Mona Lisa in Busan Beach”“AI will not replace humans, but the people who use AI effectively will outpace those who
do not. Generative AI stands as a testament to this truth, transforming industries andaugmenting human capabilities in unprecedented ways.”- Naufal x ChatGPT26

27