Master the Art of Model Compression with Knowledge Distillation | Future of Model Deployment
Channel: DataTrek
Date: 24/08/2023

#machinelearning #deeplearning #artificialintelligence 

Slides: https://data-trek.blogspot.com/2023/09/master-art-of-model-compression-with.html
Kaggle Notebook on Knowledge Distillation Training a student model to mimic Teacher model: https://www.kaggle.com/code/abhishekmungoli/knowledge-distillation
Kaggle Notebook on Data Preparation to create a dataset for training the embeddings: https://www.kaggle.com/code/abhishekmungoli/product-embeddings-data-preparation
Kaggle Notebook to train the embeddings and doing ANN to find similar products: https://www.kaggle.com/code/abhishekmungoli/product-embedding-disk-datasets
Train your own Product Embeddings | Embeddings across Domains | Training on 1.02 Billion rows on GPU: https://youtu.be/DN4S96oHRhE
BERT Demystified | Leveraging Pre-trained Bert to generate Entity Embeddings across domain | NLP LLM: https://youtu.be/v-0J7o-nDBE
How eBay Built a Next-Generation Language Model to Revolutionise Online Shopping | Bert NLP ML AI: https://youtu.be/h51nbWr7feo

In this video, we explore the concept of Knowledge Distillation, a technique used to transfer knowledge from a large neural network to a smaller one that can be practically deployed in real-world scenarios with limited resources. We dive into the different types of Knowledge Distillation, including Response-based, Feature-based, and Relation-based knowledge transfer, along with their respective loss functions.

We also discuss the different methods of Knowledge Distillation, including Offline Distillation, Online Distillation, and Self-Distillation, and their practical use cases. Finally, we showcase a practical example of Response-based knowledge distillation, where we compress a large model into a smaller one and compare it with a model trained from scratch without Knowledge Distillation.

Through this video, you will gain a comprehensive understanding of Knowledge Distillation and how it can be used to compress large models, increase the speed of inference, and improve the overall performance of your neural network.

#datascience #machinelearning #statistics #deeplearning #programming #python #datatrek #youtube #interview #interviewpreparation #interviewquestions #datascientist #dataanalytics #machinelearningengineer #datasciencejobs #datasciencetraining #datasciencecourse #datascienceenthusiast #career #careeropportunities #careergrowth #careerdevelopment #datascienceenthusiast #interviewing #ml #ai

About DataTrek Series
https://youtu.be/YvUMvFVwC_E

Business Enquiries: collabwithdatatrek@gmail.com
Find me on Instagram: https://www.instagram.com/simplyspartanx/




Music: https://www.bensound.com/royalty-free-music