RULE: Captions should be split into individual caption groups such that whole phrases, nouns, sentences, and flow of dialogue is interrupted as minimally as possible, while abiding by the 60 character limit.

WHY: This allows the captions to be easily read by the viewer, with logical breaks and enough time on screen.



Split captions…

after terminal punctuation
before conjunctions
before prepositions: Moving on to chain-of-thought prompting. Now, this isn't actually the first time we've talked about chain-of-thought prompting. We actually brought it up in an earlier section when we were looking at our RAG and our agent prompt a few lessons ago. But to define the concept a bit more formally, chain-of-thought prompting forces an LLM to generate a reasoning for an answer alongside the actual answer itself. And the goal of chain-of-thought prompting, like most prompting techniques, to be fair, is to lead to a more actionable and, frankly, just better, more accurate result. So to see this in action, we'll take a bit of a history lesson, because this is a model that you can no longer actually use on OpenAI's playground. But because I took the screenshot over a year ago, we can at least see the differences between then and now. So we're looking at a version of GPT-3 called DaVinci. There are similar models to this that can use that OpenAI claim are just a little bit better. But just to show you what GPT-3 was doing in circa 2023, if you were to ask GPT-3 a question that involves some kind of mathematical reasoning, like, you know, a store sells pencils, there's 12 pencils, if Mary buys three and gives away eight, pretty basic grade school math, and you give the AI options to pick from, A, B, C, or D, you can either simply ask the model the question, like I do at the top here, where GPT-3 doesn't really try to reason anything. It just answers the question, D, which is incorrect, for what it's worth. If you change the prompt slightly, don't change anything except adding reason through step by step, which I have bolded here. And note, the bolding is just so we can see it. It doesn't actually go through to the model itself. And instead of putting answer colon, I write reasoning colon, again, signaling that I want it to give me some reasoning and not the actual answer itself, using prefix notation, it will actually talk itself into the right answer, A. So this is obviously one example, but the concept is pretty clear. If you ask the AI to generate reasoning, it will be more likely to get the right answer. Now, we've talked about this before, but remember that order does matter in your prompt. If you ask an AI to provide an answer first and then provide the reasoning, you're going to be pretty out of luck because you're completely negating the whole concept of chain of thought prompting. Remember, autoregressive LLMs speak in order. They speak one token at a time. If they answer the question first and then give reasoning, they can give the wrong answer and then either hallucinate a reasoning that would justify the wrong answer or provide the actually correct reasoning and end the statement by saying, so I guess I was wrong when I gave the answer originally, because it can't go back and change its answer. So when you do chain of thought prompting, make sure you provide or ask for the reasoning before you ask for the answer itself. Now, more modern generative models like GPT-4 and chat GPT, i.e. GPT 3.5, more often than not, don't need to be asked to provide their reasoning first. They have been updated to be able to provide that reasoning without being told to do so. So if I ask the same question and do not ask it to provide a reasoning step by step, it still does it for me anyway. This is because as OpenAI realizes that chat or chain of thought prompting is really, really helpful in getting their answers more correct, they updated their training data to be able to influence the model to provide that chain of thought more often. Now, the push and pull of this is as consumers, it might come to a point where we're saying, well, the AI is talking way too much. Just get to the answer. Just get to the point. The problem, though, is AI engineers will know that, well, if you do that, you run the risk of getting the answer wrong more frequently. So it's either wait for the AI to finish its chain of thought prompting and then give the right answer or risk shortening that chain of thought and risk getting the wrong answer more frequently. Of course, you also, if you're using an API here, you have to pay by the token. So as you're generating a reasoning, you are still paying for each of those output tokens. So the longer the AI reasons, the more you pay that company. So we're going to take a look at a little quick case study. Well, quick in the sense of I'm going to show you the results pretty quickly. But if you want the full results, you'll have to take a look at the code in our repository. Because what I wanted to do was really test out a bunch of different prompting techniques. Because as we've now seen so many prompting techniques, chain of thought prompting, few shot learning, prefix notation, the just ask principle of trying to keep our prompts as short as possible. And we've even also seen semantic search, which we'll talk about in just a moment again. What if we had to combine a lot of these techniques together? More often than not, the best prompts out there, and by best, I mean both performant and stable, meaning as the models get newer and newer, they're still the prompts themselves are still going to work. Or if you want to move from model A to model B across providers, say from OpenAI to Anthropic, that prompt scaffold, the structure will help the AI get the right answer no matter what. The best prompts out there generally use multiple techniques in one. They have both few shot learning and chain of thought prompting. In fact, they would have chain of thought in each one of the few shot examples. So to test this, we're going to take a math question and answering data set. You may be familiar with some benchmarks, like GSM 8K, which is an 8.5,000 grade school math questions. We're going to use a similar type of data set called Math QA, the link to which you can see here. Math QA is a large scale collection of math word problems. This is actually taken directly from their site here. It includes questions, multiple choice, rationales, our chain of thought, and the actual correct answers annotated with different formats for answering those questions. The basic premise is we have a question, a train running at a speed of 48 kilometers an hour, blah, blah, blah, a rationale, a reasoning, a chain of thought, and then the actual answer given the options, option A, B, C, D, or E. The correct answer is C. So this is all we're going to need to do our test. Now, we're going to try multiple techniques, like few shot learning, how many shots should we do, one shot, two shot, no shots, zero shot learning. Do we include chain of thought? Do we not include chain of thought? And then we're also going to add in impromptu prompting technique, which I'm going to call semantic few shot learning. The idea here is in few shot learning, I never really told you how to pick those few shot examples. The easy way is simply to pick random few shots from some reserve data set and say, well, these should be similar enough to show the AI what I'm looking for, the format I'm looking for it in, and to improve the model's performance. And as we'll see in just a second, that does improve the model's performance. But what if we could make those few shots even more useful? So for example, what if we had a question about, in this example, an Easter egg hunt, and instead of just picking random questions from a data set, what if we could find other similar types of problems that have a similar format, have a similar scenario, have a similar situation, and use those instead of just random few shot examples? Huh, if only there was some way of easily finding semantically similar items from a data, oh, I remember semantic search. So if we use semantic search, embed a corpus of questions that are not going to be asked at any point during the testing set, i.e., embed the training set questions, if we can dynamically pull in examples that are semantically similar to the one at hand, will that make the system even better if we're including more relevant few shot examples? The short answer is yes. So what we're looking at here is really just a snapshot of the test itself. The test, which you can see the full code on our repository, goes through several different closed and open source models, including two that you see on the screen here, LLAMA3 8 billion, which is on the bottom right, and on the top left, we have Anthropx Opus, which is, as of today, Anthropx's largest and most performant model. Now, the goal for each one of the models is to test a few different prompting variants. Well, to be clear, you can see them on the screen here. I have 10 different prompting variants. I'll go through them really quickly, from left to right on the screen, for, I believe each of the models should be exactly the same. Yes, they are. So from left to right, we're going to have just ask with no shot learning and no chain of thought. That's simply inputting the question into the AI and saying, give me an answer. That's it. No few shot learning, no chain of thought that we asked for. The next one is going to be same thing, just with chain of thought, asking to reason through it step by step. Then we're going to do a random one shot learning with no chain of thought. That's the third one. The fourth one is going to be a random k equals one with chain of thought. The fifth one is going to be a random three shot, no chain of thought. And then, you guessed it, a random three shot with chain of thought. Then we're going to get to in these gray bars, starting right here. Then we're going to start getting to one shot learning, but with semantically similar examples, no chain of thought, semantic one shot with chain of thought, semantic three shot, you guessed it, without chain of thought, and then semantic three shot with chain of thought. Now you can see for both models, we can see an improvement in performance by adding prompting techniques. For both WALMA and Anthropx OPUS, OPUS starts at about 50% accuracy just by asking a question, which is fine. It's a coin flip on whether or not you can do a word problem. But by the end of our prompting, we're going to get about 75% accuracy simply by prompting alone. No fine tuning. This is semantic few shot learning with or without chain of thought. Now, funny enough, for OPUS, we see that the with and without chain of thought function roughly at the same performance. You can see that the non chain of thought performs slightly better than the with chain of thought. That's not going to be so common for models of that size that are, again, relatively speaking, that capable. The chain of thought prompting may or may not actually provide a huge boost in performance. But for WALMA, it did provide a big boost in performance. For WALMA, the best performing model, which to be fair is roughly 40% accuracy, is with semantic three shot learning and chain of thought. Without chain of thought, the performance drops to 30%. This is why we test. We have a later section dedicated to testing LLMs, but of course, there is really no talking about LLMs without testing them along the way. So here we have two models being tested across these prompting techniques. It's also worth noting, to be clear, that these kind of like this up and down scenario that you kind of see here, up and down, up and down, that's because we keep adding chain of thought and then removing it. So in the first one for WALMA, let's say, we ask the question, no chain of thought, no shot learning. And then we suddenly ask it to reason. And it jumps from under 10% to roughly 25%. But then we take away the chain of thought and add in an example, which does slightly better. It's hard to see here, but does slightly better than just asking with no examples, but significantly worse than with chain of thought. It's adding the chain of thought that tends to add the boost with and without. This is true mostly for OPUS too, adding chain of thought seems to boost the performance, except when you get to the larger, the semantic examples, the semantic examples are still providing themselves a boost, but the chain of thought seems to not be helping out as much as it was with WALMA. Again, testing is always going to matter. Last thing I'll note is if we look at the difference between semantic and random, that's also going to be quite stark. So for example, let's look at this one with the stars, this third bar here, this is a K equals one random, this third bar here. And then, if we look at semantic K equals one, that's this orange bar over here. There's a big, relatively big difference between these two bars. So adding semantic examples clearly seems to help. Again, you can see the full results in our code, but for almost all of our models, in fact, for actually every single one of our models, except OPUS, which has this kind of weird fluke here, the best performing prompt was always semantic K equals three with chain of thought. OPUS is the only one that barely beat it by no chain of thought. But again, we've already explained why that might be. So in the end, prompting not only works, but a combination of prompts ends up being some of the best scenarios here. By adding chain of thought with few-shot learning, clearly makes our models more performant, at least on this testing dataset.