#38 Back to Basics  RAG  Transformers  ML Optimization  and LLM Evaluation.
Source: tai_blog

Good morning  AI enthusiasts! This week  the community and I are answering some recurring questions about RAG  coding assistants  transformers  machine learning  and more. You will also find fun collaboration opportunities and memes.   Enjoy the read!   Whats AI WeeklyMany clients asked us (Towards AI)  But why would I use RAG if Gemini can process millions of tokens as input? So  is RAG dead? Thats what I investigated in this weeks iteration of Whats AI. I explore the differences between RAG and sending all data in the input and explain why we believe RAG will remain relevant for the foreseeable future. This post should help you determine whether RAG is suitable for your application. Read the complete issue here!    Louis-Franois Bouchard  Towards AI Co-founder & Head of Community   This issue is brought to you thanks to GrowthSchool:   U+1F9BE Master AI  ChatGPT and 20+ AI Tools in just 3 hours   Dont pay for sh*tty AI courses when you can learn it for FREE!   This incredible 3-hour Workshop on AI & ChatGPT (worth $399) makes you a master of 25+ AI tools  hacks & prompting techniques to save 16 hours/week and do more with your time.   Sign up now (free for first 100 people) U+1F381   This masterclass will teach you how to:   Do AI-driven data analysis to make quick business decisionsMake stunning PPTs & write content for emails  socials & more in minutesBuild AI assistants & custom bots in minutesSolve complex problems  research 10x faster & make your simpler & easierYoull wish you knew about this FREE AI masterclass sooner U+1F609   Register & save your seat now! (valid for next 24 hours only!)   Learn AI Together Community section!Featured Community post from the DiscordAman_91095 has been working on the GenAI Career Assistant  built using LangChain and Streamlit  a project designed to experiment with AI-powered job search tools. It helps with the job search process  helps you find job listings that fit your profile  generates cover letters customized for specific applications  and provides useful information about potential employers. Check it out here and support a fellow community member. If you have any feedback or questions  share them in the thread!   AI poll of the week!The results show a very clear reliance on ChatGPT. Are general-purpose models enough for most use cases? Are specialized models only required for proprietary applications? Lets discuss this in the thread!   Collaboration OpportunitiesThe Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI  want a study partner  or even want to find a partner for your passion project  join the collaboration channel! Keep an eye on this section  too  we share cool opportunities every week!   1. Ananya.exe is looking for a partner to collaborate on a finance-based project (which involves knowledge of multi-AI agents  RAG pipelines  information retrieval  NLP tasks  end-to-end development and deployment  etc.). If you know finance and can work with the above technical specifications  reach out in the thread!   2. Gere030199 is seeking a marketing co-founder for their Discord bot project. They need someone experienced in creating engaging content. If this sounds interesting  connect in the thread!   Meme of the week!Meme shared by creitingameplays   TAI Curated sectionArticle of the weekStreamline Your LLM Evaluation: A Step-by-Step Guide to RAG Metrics with Streamlit by Maxime Jabarian   This piece presents a new Streamlit app intended for RAG evaluation. It offers an easy-to-use platform that shows chatbot performance using clear metrics and graphs. By integrating a comprehensive set of evaluation metrics beyond simple accuracy  the app ensures that users can easily understand and interpret the strengths and weaknesses of their LLM models in a clear and visually engaging manner.   Our must-read articles1. How to use SVM in Power Systems Analysis? by Optimization team   Machine Learning has become a buzzword lately  with recruiters frequently advertising data scientist positions when theyre really seeking experts in optimization. This post emphasizes that many machine learning methods are fundamentally based on optimization. In other words  optimization laid the groundwork for the development of machine learning  much like the chicken laying the egg!   2. Attention is all you need: How Transformer Architecture in NLP started by Surya Maddula   This article discusses the evolution of transformer architecture in NLP  starting with the Attention is all you need paper. It also explores the problem of contextualized word embeddings and how transformer architecture addresses it by introducing the encoder-decoder model for translation. It also presents a few fine-tuning examples and transformer-based language models.   3. Querying SQL Database Using LLM Agents  Is It a Good Idea? by Sachin Khandewal   This blog explains different ways to query SQL Databases using Groq to access the LLMs. It also explains how to leverage LLM Agents to build an SQL Agent using an advanced DSPy framework and highlights its limitations.   If you are interested in publishing with Towards AI  check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.