Summarise the following transcript: david: So for the people that might've missed this we've launched this initiative where we will translate Validated high quality prompts from English to different languages. And we've asked the Agilea community and the Hugging Face community to take the lead in terms of leading these translation efforts and communicating and all these kinds of things. So I think gathering, yeah, more and more high quality prompts for different languages, because sometimes these automatic translations, it can be off. And yeah, that's one of the reasons that we. Yeah. I want the community to help and pitch in and yeah, Krishna has offered to, to lead a handy example. So if if someone might be interested in that, feel free to pitch in and go to the hugging face discord. Go to the data is better together page and then you'll find it there. If anyone wants to do a brief introduction, feel free to say hi, or maybe tell us something, what you're working on, but otherwise I will maybe ask Elad Levi to give a brief introduction about what he's been doing on intent based prompt calibration. How that's, how that works. It was translated in a practical way into the super cool out of prompt open source project that you've been working on together with three friends, I believe. And how you actually built that on top of Argella and what's the idea behind that. So feel free to, to give a brief introduction. Glad to have all of you here and then I'll give the mic over to Elad. So you can start if you want. elad: Great. Okay. Thank you very much. Okay. So what we will talk today a little bit. So I will first describe the challenge of prompt sensitivity and I will show some very surprised results of how much large language models are really sensitive to the prompt. Then what I will describe is the previous methods for prompt optimizations and the issues that they have. I will then describe our contribution, our methods and how we tackle these issues. And then we will discuss a little bit about the auto prompt. So we build an open source system that implement these methods. And using also Argila as a UI server. So it's like a really cool system. And we will talk about the system and we'll show some demo. So about prompt sensitivity. So you have these two prompts. So in the first one, it's almost exactly the same phrasing. Just is does instead of is, adding some review. Cool. looking exactly the same. So if you will ask some human annotators and give it as an instruction to the annotators, I'm sure you will get the exact same results. There's no really difference. But when you're asking a large language model, you are seeing here that that the there is like very High discrepancy between the results. So we prepare some spoiler benchmark. It doesn't really matter what exactly was was this benchmark, but it's contained about 500 samples. We run these 10 times, so we get statistical significant differences. And you can see that there is a huge gap between the performances of the two probes. So GPT 4, which You think about it as state of the art very similar to human, but he perform significantly different on these semantic differences. Now, another thing that many people's are doing is they take this prompt and they like GPT 4 suggest a prompt. So we instruct GPT 4 to take this initial prompt and generate some prompt, which is better. And you can see that it looks really nice. You can see that is really describing the task of spoiler classification. And he add description of what is spoiler. So it's looking it looks amazingly well, however, the results are even worse than what we saw earlier. So there was no improvement. And although it looks good, the results are not great. So even if you if something is look good. When you evaluate it, the performance might be not that great. Another issue that we try to tackle is the prompt ambiguity issue. So when you have a prompt, you can see here the example of advertisement classification, yes or no. You describe the prompt in a few sentences. So always the ambiguity of the language. Of what to do in all the edge cases. So you see here the example of describing some situation of purchase. Some of the people will say that this is advertisement, and some of the people will say that this is not advertisement. There is this ambiguity. And this is something that we want to to resolve this ambiguity issue of the prompt itself. Now, the last thing that I want to explain about the prompt sensitivity is the changing in the behavior over time. So even if you have found a great prompt, You are satisfied with the results. OpenAI is changing constantly the version. They want to update the model, the knowledge base of the model. And you can see here that there is huge discrepancy between the results between the different versions, both with respect to the accuracy of the model on some tasks. So you see the GPT 4 has decreased accuracy on this task and GPT 3. 5 actually increased the accuracy. So it's really flickery, but also with respect to the behavior of the model, which is like even more important. So you can see that less stop apologizing or other behavior, for example, verbosity. So the behavior of the model constantly change and you need it to all the time update your prompt accordingly. So the conclusion is that we cannot try the prompt engineering process, the regular prompt engineering process. And we need to do prompt engineering correctly, doing some optimization, methodology correct. And there are a few methods that suggest how to do it. And I would like to describe a very recent work, which is called OPRO. And what we did, they did in this method is a very simple process. So they start with some initial prompt. Let's figure it out. So they try to solve some mathematical questions and let's figure it out. What's the instruction? It's like a chain of, you can think about it as a rephrasing of a chain of thought. And then they evaluate the score of this initial prompt. Okay. So they have the benchmark. They use a well known benchmark of GSM 8K. Okay. Okay. And you see the score 61. Okay. So the evaluation is done according to the benchmark. And then they took the prompt itself and the score and they provide it in, they put it in some meta prompt and the meta prompt described the task the task that need to be done. So he described that you have Few prompts and they score this data score and he asked GPT 4 in this case He asked the model to provide a new prompt which has higher score. Okay, so you provide few Samples and they score and you ask the model itself to provide a new prompt which has better score and you could Provide him also like few examples of the prompts in which there was a failure And then you can see, this is the new prompt that was suggested. Let's solve the problem. And now you continue the process. So you continue to evaluate this prompt. You put it in the meta prompt. And you let the model do these iterations until it gets to the calibrated prompt. This is iterative process. And you can see here the results that they tested on BigBench hard, and they got up to 50 percent improvement on 2023 challenging reasoning task. So they got some really nice improvement. So this is the OPPO paper. Now there is more recent work which called prompt engineering prompt engineer. I will not go into the full details of this work. This is work that simply did a prompt engineering on this metaprompt. We have the metaprompt that provide the new prompt. So they did just prompt engineering. many experiments. This is experimental paper. And I will not, most of the things they test actually not improved the metaprom, but they have something that we take from this paper of splitting up the to two to two steps such that in the first step, we provide the current prompt, a false example, like the error and ask him to provide some error analysis. And then we put this error analysis to the meta prompt to generate the new prompt. Okay. So this is the idea that we took and indeed, this is something that improving the results. Now, this metaprompt and using large language model as optimizer, the advantage of this approach for port optimization, that you can run it on every black box LLMs. You can run it on OpenAI proprietary model or Gemini. Whatever you like, you just use it on this black box LLM. However, the challenge with this approach that you need to iterate it with this very strong LLM, because the metaport requires strong LLM, on some high quality large benchmarks. So in order to improve it and give a good feedback to the model. You need very high quality, a large benchmark. And in many cases, this is not feasible or it's hard to get. either because there is a low resource task or the data set is a little bit biased. And of course, this is very costly. Running GPT 4 on very large data set with large amount of iteration in Opro they did About one thousand and six hundred iteration. So this is something which is very costly. It's get to up to a few hundred dollars. And we didn't have the money to do it. So we need to come with a new method which is much more efficient. So what was our idea? So our idea was very simple. You see you again, you give the model the initial prompt. Generate a spoiler movie classification, and then you see the nice UI for Agila. The model, the LLM model generates synthetic samples, so it generates about 10 synthetic samples. And you annotate them, yes, no, according to your labels. And so you annotated a few examples. And then after you finish this annotation you are letting the model optimize on top of this benchmark. So now we are using the same idea of Metaprom, where we provide this feedback, and we do the prompt optimization. So we get a new prompt, and we now generate a new synthetic data for the new prompt. Now the main idea that the synthetic data that we are generated are not like standards standard data set, but it's a boundary cases. So it's ask the cases that best differentiate between the different classes. So this is like the main tweak that we are doing. So this, since we are going on the boundary cases, we got very, distinguish very effective data set. Although we have a few samples because it can separate and isolate all these edge cases. So the idea is to calibrate according to the user intent and the user intent. You get it from these synthetic boundary cases. And here you can see the example. So what the model is getting is just the task description and the initial point. You see the the advertisement classification, yes or no. And this is the generated synthetic data that the model generate. We already saw it. It's really a boundary cases. So the model predict no, and this is a legitimate answer. But however, I was the annotator and I decided I want to be more conservative. And I told the model this example is indeed advertisement because you mentioned some company. And that in the next iteration, the model immediately understand and fix the prompt. And that he said that even if it was indirect advertisement. Oh, you just mentioned a competitor store. Then also in this case, annotated as yes. Now you see that now in this iteration, there was a synthetic data. In which there is no mentioning of of any company. So in this case, the model was over conservative. So it predicted as advertisement. Although I, as annotator, I said that it's not an advertisement. Again, this is an edge case. And after a few iterations, this is iteration six. The model catch all the nuance and get to the prompt accuracy of 97%. Okay, so he catch all these like boundary cases and was able to refine the prompt correctly. And it seems to work like very effectively. So about the evaluation process, so how we evaluate and compare ourselves to the previous method. So we want to evaluate ourselves on a real world use case, not on classic benchmark, reasoning benchmark like GSMK, because we thought that like these are not really a benchmark that, you take for, these are not real world use cases. So what We did a very simple evaluation process. We take very powerful model, GPT 4, and we provide very highly detailed prompt. You see this very high detail prompt for the spoiler classification task. So this prompt contain like all the nuance of what to do in all the edge cases, and it simulates human intent to a given task. And we use this prompt as a ground truth. So this is the prompt that was used to create the ground truth. Then we start with some initial prompt and try to converge to this prompt. We compare ourselves to the ground truth. And we compare between the different methods. We use the IMD review data set. This is a classical data set. And we test with respect to different tasks. And you can see the results. So on top, this is the red one is our method. And you can see below that this is Opro and the prompt engineering approach. And you can see that our method outperform previous methods. And you don't need too much samples with only between 30 and 50 samples. This is enough in order to get a very high quality results. And so you get to 30, between 30 and 50 samples. This is all that needs to be done. In all the cases we test on 30 iterations of the the model. So it's really not cost too much, not it's, it should not cost more than $1. This is like the target that that we, we achieve. And you can see here the example. So this is the initial prompt that we already saw that this we will contain a spoiler. You can see here that o simply rephrased the prompt. And this is prompt or indeed this is what it do is just phrasing the prompt, but it's very effective. We saw already that the prompts are highly sensitive and you get a boost in the performances. This is the prompt engineering. So indeed this is more descriptive prompt and the accuracy, of course is better. And this is our method. And you see how this is a highly detailed prompt. And it indeed catch all the nuance of the ground truth form and you get a boost in the performances you get to 92. 3 accuracy. So it's indeed improved significantly on top of the base prompt. Now, we, of course, did some ablation study, and that all the components are really essential. Of course, the synthetic data improves significantly, but what is interesting is that also the iterative process of generating the synthetic data in each iteration and not generating directly from the start all the synthetic data. This is also provide an improvement and it's really it's not surprising because in each step we prove that the problem and we want now to ask boundary cases on the new improved prompt and not on the bad initial prompt. Now, if we want to understand the effect of our method. So one of the important things that we are doing. is shifting the data distribution. So if you look at the spoiler classification task, so in the real world data set, in the IMDB review data set, less than 20 percent of the data set contain a real spoiler that was labeled as yes by the ground truth. But in our synthetic data set, We instructing the meta prompt to the model to provide a balanced dataset. Okay, so this is the part of the instruction, and you can see that indeed the data, the synthetic dataset is much more balanced, set, balanced dataset is close to the 50%. Yes and no. So this very important property of the synthetic data set, and we will see immediately the effect of this data shift distribution. So another thing we want to tackle that was not done in previous paper is tackling the generation task. So everyone working on the classification task, but Most of the real world use cases are a gener generation point and not classification point. And we suggest a new method to handle a generative task, which is very simple. So what we did, we take the generation port, so this is the generation port, generate some movie review with specific style and we translate it. To a ranking prompt, we did it by simply ask GPT 4 to generate from this generation prompt an initial ranking prompt. We already know that he knows to do it something that looks good. And you can see that he's doing some nice job. He ranked it between 1 to 5 this is the rank. And he provide a nice instruction for the ranking prompt. And then what we did. So we calibrated the ranking prompt. So we treated this classification. We already know how to solve a classification task, and we treat the calibration of the ranking prompt as a classification task. So in each iteration, we we generate synthetic edge cases. And we ask the human annotator to rank between one to five. And after we have this calibrated ranking prompt, we use it as a ranking model. So we generate synthetic samples using the current prompt that that we investigate. We evaluate using the ranking prompt, so there is no need for human intervention in the second part for human annotation. And we analyze the prompt performance according to the score. And using this iterative process, we suggest a new prompt, and now rank it. So for this second part, there is no human annotation. Only the first part. Again, you need only 30 examples of ranking prompt, and this is all you need for the generation task. So very simple. And you can see here the results. So This is the initial results. And using our method, you get to a calibrated prompt, which get almost to score a five, which is the best score. So the method is highly effective. Now what is interesting is when you apply these these methods on OPO and pe, this idea, you actually get degraded performances. comparing to the initial prompt. So you're getting worse prompt. And the reason for it is that if you are looking at the ranking Histogram of the real world of the IMD review view data set It looks like not surprisingly as some normal distribution around the mean, ranking around the tree So you have very few very few examples which get to the five score you see here very few examples. Now so now when you are fitting on top of this data set, the ranking problem, you will not able to distinguish correctly between four and five because there's not enough samples. So your anchor that you fit is not is not properly. Now of course the generation of the review of GPT four is at least something between four and five. It will not get improved, it will get worse because it doesn't know how to separate well. But remember that we are having the power, the synthetic data, to shift the distribution of the samples that were generated. And we ask the model in the metaprom to provide boundary cases between rank 4 and 5. And you can see the distribution here. And now when I'm fitting a ranking function on top of these, this data, he will be able to distinguish well between rank four and five. So the second phase in which we optimize the generation part is much more effective. And this goes to the boost in the performances. So this is like really nice results that we have. So now a little bit about Autopront. So Autopront is a system, an open source system that we build which implement this method. And it's nice because the system is really modular, and you can replace each component very easily. We already have very large users and companies that are using this system in production and got amazing feedback. So the system is composed by three main components. There is the dataset, which is connected, of course, to the database. But there is also some semantic DDoP in order not to generate synthetic data that are too much resembled to each other. So We also do some semantic deduping. The main class is the estimator class, and it is either and you can use it as the annotator and the predictor. So the annotator is the part that is responsible to generate the ground truth. And there is the predictor that is all that this is the optimization part that you are optimizing. And the ator can be connected to our gi a server with the amazing ui of our gi a and the predictor is, again, connect to the LLM, but you can change it. You can plug the annotato annotator to the LLM service, and of course we have the evaluator. And you can modify easily the score function. So the evaluator get the prediction and annotation and decide what is the score, how to compare between them. So what we can do with auto prompt, so we can do besides prompt optimization, which is the reason for the system. There is like also nice things that can be done with the system besides the productization. So one of them is the prompt distillation. So if suppose you have you are happy with the prompt in GPT four, and you want now to save cost and you want a GPT three prompt that, keep the performances of GP, GPT 4. So we call it a distillation. We want to distill the knowledge from GPT 4 to GPT 3. So what you can do, you can easily use The GPT-4 prompt as the annotator and train the GPT-3 as the predictor, and he will try to fit the performances of the GPT-4. So this is really nice. Another things that, that we are doing. It's prompt squashing and prompt batching. These are methods to squeeze your prompt. So if you have multiple prompts and or you want to send multiple samples in the same prompt, we are doing also this optimization. And again, this is for saving a cost. It's really helping in saving the cost. And of course, we are also, you can also use the report just to generate a synthetic benchmark and use it. Only for this purpose. And okay, so let me show you some example. And so I hope that you are seeing my screen. So in order to run, you simply, you can use, we have the run pipeline. You just need to write the prompt, the initial prompt and the task description. And you run it and you get to the, It will pop up in our Gila UI. You can just annotate. Yes, no, yes, no. And the moment you finish to annotate, you have your 10 example. You are seeing here okay. You are seeing here that it start to there is a listening on the Argilas server side. This is the implementation that we have. And the moment you finish, he process the sample. And in a second, he will suggest you see you got a new prompt. Of course, accuracy is hard because I annotated randomly. But you can see that it start generating a new prompt. Now, if you want, for example, to use LLM and not using an argila, so the only things that you need to do, so you see here, this is the annotation, the annotator the annotator configuration when I use the method argila, and I use the config file, And in order to make it some to replaces with LLM, I just need to replace the annotator with LLM. You can see here I'm using Google Jimini Pro. I have some instruction. This is the prompt that is used as the ground truth. So let's go to the run pipeline. And switch it to the to this annotator. And he will now do the iteration without using the the human annotator feedback. He will just compare himself to Germany 4. Let's wait for a second until we see okay, so we take so here you see generate the initial sample. It takes a little bit of time because we are asking a GPT 4 model and it's a long prompt, so it takes a few seconds. Here we finish here we finish the initialization. And now you can see that it is running Gemini. without using the annotation pipeline. So you can see that he's run, he's processing the sample, now he creates the annotation. In the second step, now he predicts, so this is GPT 3 and a half, so it's much faster. And now he will suggest a new prompt. So it's very simple. You don't need the human intervention in this one. So you get like this new prompt and it will get improved and improved. So to conclude, Okay, so elad: the conclusion. First of all, optimizing your prompt is a must. You must use a valid benchmark. I think this is like the main message from this presentation. Whatever method you use some benchmarking. Don't trust that something looks nice. The second thing is that LLM can do anything. They can generate synthetic data, they can do optimization very well, they are a highly effective tool that you need to use. And of course, Autoprompt is an open source. You are welcome to use it again. We have many companies that are already use it. It's completely free and open source and you can try and test it. And I encourage you. Thank you very much. david: Cool. Thank you for the presentation. It was really cool to to see and to hear in practice, and I think we got quite a lot of questions. So I'm going to try if you're ready for the fire. So maybe the last one that I I asked was, do you see any difference between the human and AI feedback during the optimization? So is the human feedback normally better in terms of generalization or in terms of intent based pump calibration? It's. On par. elad: So I usually, yeah, for sure. Human feedback is better when you have, most of our instruction to annotators are not we are some pro we show them an example. We instruct them very well. We do few iteration. And this is something that that you are not doing with the letter. You just trying to. Do some prompt engineering and optimize it. And this is the main advantage of using human annotator. Now, the other advantage is cost and efficiency. I'm using also LLM as annotator in some cases. But, for most of the tasks, I usually prefer to do it like human annotation. david: For most production cases. Okay, that's good to hear as also one of the things that Danny often mentions like our CEO that yeah, you really need to dive into the data to really understand what's going on, even though optimization to a certain up until a certain point is very useful and convenient, of course. And how do you see something like, for example for a basic classification example given that you're doing annotation, how do you see this approach differ compared to something like setfit? Because if you're annotating examples, you might be able to use those synthetically generated Entries as a way to actually train a set for classifier instead of using like LLMs to do that. elad: So our system actually support using using your own custom data set. I think that if you, the synthetic data is making it much more efficient and it's able to if you have a few hundred annotated samples on your data set, and yeah. You are fine with the cost of doing like this iteration on this data set then. It's very good approach. And we have actually spent last three meetings with the, some companies that wants to help with auto pom to to remove the synthetic part. They will just want use the, want to use the optimization, and this is. For sure, but in many cases you don't have this data set and you want to do it like more efficiency, efficiently. And for this, like you, the, I think the system is like very very well, very optimal. david: Okay, cool. And in terms of the synthetic parts, then you would replace it with your own data set and then just sample random samples from there and then iterate based on those. Okay. elad: Yeah, so we have the option also to insert a few samples of your data set, and when we generate synthetic data, we are not generating them from scratch. We are using these examples of the real data set and generate samples which are resembled to this data set. In order to not have some distribution shift. So this is very important. And in the I'm the big review that said we did it. We use about 10 samples off real reviews because otherwise, when you're generating from scratch completely, And then for sure you'll get suboptimal results because the model don't know the real world distribution. david: Yeah, that makes sense. And in terms of you mentioned forms calibration being able to use for generative tasks as well. Do you see any differences in for example the number of rankings that you force the model to define and some other differences in terms of how you would be able to use this in an optimal way? Because within this label, we are doing a lot of these Data set generation tasks. So it might be very interesting for us whenever we create a synthetic data set to actually use this. So what would be best practices in that sense? elad: Yes, I think that it's very challenging to annotated ranking task. I tried to do it and it's not easy to separate between this delicate separation between three and four. So I What we will do like we are trying we will do it like we will add it to the repo soon We want to add like few not one rank because this is challenging, but many attributes and rank it according to different labels and attributes And I think that this will be like a real Improvement to the current system because the current challenges the current challenge is that it's not easy to human to annotate properly and consistently. between one to five. It's not that easy task. david: Yeah, we have seen something similar with the data is better together efforts and these kind of things. And yeah, and the data are also different than even in the morning or in the afternoon, people tend to be more. Yes. And what's the trade off in your opinion in terms of having the potentially additional front length after prompt calibration or optimization versus a potential marginal gain in accuracy of the model elad: compared to david: what? Maybe the marginal gain in accuracy. So if you don't gain as much accuracy improvements. Based on the after the optimization compared to having the additional costs, so to say of having a longer prompt in, in terms of calling your LLM using your element in production after. elad: So there are like two. We want to actually important engineering. They have the ability. They instruct the model to create very compressed from and limited the number of tokens regarding the optimization process. We have, of course, like some stop criteria and we can, you have, you can limit the budget. So you can say that you want to optimize you have $20. So this is really nice. You can actually put the number of dollars that you want to optimize and optimize until like this stage. But of course we have some stopping criteria. If there's no improvement after fuel iteration, then the model stop to iterate. And you are not, david: okay. And yeah, feel free. If people have other questions in between, feel free to post them in the chat. Then I'll answer them later or leave viral. Sorry. And Chris had one question about is what's for you are goods. Number of examples to use a seed for the synthetic data generation when the task is a bit more niche for example, a medical domain or the finance domain, which might require a bit more funds. elad: Yeah, it's a good, from our experience again. using some 10, 20 real examples, and then let the model to generate about another 30. It's working great on many of the productive use cases that we saw. But of course, elad: We need. I need to look at the task in order to say exactly what is the number. And I'm suggesting to simply play a little bit with it. There are very few hyper parameters that you need to play with them. But again, the basic recipe, will work, I think very nicely also on this child david: yeah. So I'm not sure if you have any more comments from your side. Otherwise. Given that there are no more questions, I would like to thank you for the presentation. I really enjoyed it and I'm sure we will use the framework for some optimization with this label and potentially it might be interesting to see if there's a an integration possible. Yeah, as always the slides will be shared along with the presentation and everything will be uploaded for the people that maybe joined later. Keep track of YouTube and LinkedIn and other socials. And I would like to thank everyone for coming and then hope to see you soon to during one of one of our other meetups or somewhere online. So thanks. Thanks a lot for coming. Thank you very much. Goodbye.