Why is Self Attention called "Self"? | Self Attention Vs Luong Attention in Depth Lecture | CampusX
Channel: CampusX
Date: 2024-03-13

The term "self" in "self-attention" refers to the fact that the attention mechanism is applied to the same sequence or input, rather than relating an input to an output sequence. In other words, self-attention allows the model to learn about the relationships and dependencies within a single sequence, such as a sentence in natural language processing or a sequence of images in computer vision. This is in contrast to traditional attention mechanisms, which often involve conditioning on the output sequence or comparing the input to a separate reference sequence.

Digital Notes for Deep Learning: https://shorturl.at/NGtXg

============================
Did you like my teaching style?
Check my affordable mentorship program at : https://learnwith.campusx.in
============================

📱 Grow with us:
CampusX' LinkedIn: https://www.linkedin.com/company/campusx-official
CampusX on Instagram for daily tips: https://www.instagram.com/campusx.official
My LinkedIn: https://www.linkedin.com/in/nitish-singh-03412789
Discord: https://discord.gg/PsWu8R87Z8
E-mail us at support@campusx.in

✨ Hashtags✨
#SelfAttention #DeepLearning #CampusX

⌚Time Stamps⌚

00:00 - Intro
01:40 - How is Self Attention is an "Attention" Mechanism?
03:30 - Quick Recap
12:17 - How is self attention similar to Bahdanau & Luong Attention?
21:27 - Why is it called "Self" Attention?
22:00 - Outro