I created a new agent the reason why I haven't released it is because I think it's so powerful that I'm actually kind of worried to release it and I don't know like if people might misuse it that is Petro the creator of two AI Asian teams Maestro and Deli the second of which was never before seen until now my name is <PRESIDIO_ANONYMIZED_PERSON> and this is a demonstration on how to use Maestro and Deli this is straight up from the gethub R so this has no changes there's no like it's not like a special but this is ex exctly like what everybody will download on yeah I I'll link the repo below basically at the start you have all your sort of like you know different models and so basically the way it works you have one main orchestrator that's like the smart model which is in this case I'm using oppos right that you specify a goal you specify this goal and then you will basically you know it will break it down into task but the difference that myor has compared to any other of the previous sort of like you know BB AI or Auto GPT all those like older agent framework is that it doesn't create the task all at once instead what he does it creates the next task and then after the sub agent which in this case is Sonet but you could replace that with IU or another oppos it just you know it just depends of you how fast you want how much money you want to spend right yeah the nature of the problem trying to solve and then after Sonet responds to the first task then Opus refines what the next task is going to be so the beauty of it is that it's actually much more dynamic because he deciding the task in real time and by doing so you end up I think with a better results and I think you know we'll see in a second like the kind of stuff that you can get out of it and the other thing too the model basically each of the orchestr model and sub agent model they have the full knowledge of any of the chat that has happened in Fr the start and that's also the beauty of using uh Cloud because I think it's around like 200,000 token context so like you know you you get a lot of information baked into that and so so the AG are less confused right like when you give it less confuse it knows the context of the conversation and because he knows like the code that you know the agent before it decide that is not makinging stuff up that wouldn't work with that code right yeah and and so and this is is the thing that I wanted to show you that cloud made for me which is kind of crazy to think about it now right so basically Cloud was like okay uh you know based on the following objective which is you know it's the one that the user specify and the previous task if any right and of course I told him hey this is how it should work but then he come up with the the system problem please break down the objective into next sub task and create a concise and detailed prompt for a sub agent so that he can execute that task I this part because it's always good to use like um um you know like remind like yeah to just know that you know this thing is important when dealing with code make sure you know that you fix the code if needed and then if you find any bug you know provide a better code blah blah blah and then basically there's like this entire section here we don't have to go into the details is about like how to structure the the the file and the code information and then basically what happened is like that's the orchestrator right and then the aging it just it takes all that context and it just keeps keeps going at it right like okay I answer this question they asking next question asking the next question and then the sort of like the final um refiner right that's like the oppos refiner that one what that does takes this entire conversation that happens between these two agents or I guess multiple agent because it's like one after the other and it Con it takes all the data all the files and he actually creates the file and the folder which is pretty amazing right and so let's just go now and I'm just going to like run it right so basically the way you would do it you can either run it into like your corer app like your Ida but if you don't feel comfortable with that you can also just have it in a folder this is like for the people that want to use it for the first time and then you can just say you know new terminal of folder so I'm going to delete this one and if you have python install in my case I have Python 3 so I'm going to say Python 3 uh and then the name of the uh script which in this case is myestro classic and I call it like that just to you know to differentiate which one is using uh opposite which one it doesn't and so now what that does is ask me for an objective and so the beauty of this I can actually put a file too like if you have a file like a text file because the what myestro does doesn't only work for coding like you could potentially like give you know like 18 pager of something like a PDF or you know something as a as you can fit fit so much into the 200k it's crazy exactly right and so you can give a lot of information but in this case we'll just give like a simple objective so I'm going to say hey create uh paint like up in and using HTML CSS and GS let's just try that and so I also add search which is pretty cool I'll show an example later which actually what it does the each of the agent is connected to the internet and is looking for the answer online but in this case we don't need it because you know it's just a very simple question and doesn't have any like uh recent knowledge in this case right so you will use the search if you need something that's like happened today or in you know in the next week or something um so I say you want to search I search and so now the first the oppos what it will do it will basically yeah call these objectives I'm GNA like Zoom this up a little bit and could you please zoom in even more like a lot of people watch on their phones as well so okay so see this is the first task subop task set up the HM and the CSS St for the paintt to the op include a canvas a color keeper but so he decide all this stuff which is kind of crazy if you think about it right and so now son it right is is say oh okay this is the HTML and the CSS right what do you think and now the orchestrator you know like sees that goal and if he has a if if the first task is complete he will say okay this is complete if it's not it will like you know go back to it and and and like kind of like refine it and so like the beauty of this is like you know Sonet is not only cheaper but also faster right so like you don't have to use Opus for everything and you save money and you get it done faster exactly and and to be fair I could have used IU too but I want to show you know like like what I think like people will use it the most which is the combination like oppos uh Plus on it and I also add a feature where you can actually see a much money so it's just nothing that's really useful z.0 right and so you get all the all the all the cost here oppos of course is like a little more expensive um that's super useful and so now it's going right so now it did the HTML it did the CSS and now he's doing the JavaScript for it right and so he's just like it's just like going for it by the way I see a lot of comments asking me whether I do Consulting and I do however I charge $500 per hour however if you're inside of the community you get three weekly calls each one hour where you can talk to me directly and ask me anything so if that sounds interesting to you then make sure to join it's the first link in the description I also add another thing because the maximum amount of token that each of those molec sped out it's 8,000 token and so basically uh if it happen that it goes above 8,000 token which is very unlikely but if it does happen it just creates uh you know two um two different uh prompt basically sorry not prompt results and then those just get included in the in the loop basically yeah uh now he's going you know he's like oh I like what you're doing why don't you add the so this is the this is why I loved like my estro because he gave me a way into looking how this thing is thinking right like I didn't ask to add Zoom GRE overl I mean it's going like a little like crazy here right like like you know doesn't have to do that but because we went that specific right I could have say like a very simple app that just has to do this yeah right but we give it a very sort of like empty feel right and so of course he's going a little more more like kind of you know on the off off off the line but it's also cool to see how the things hopefully it wouldn't like be too long um maybe you know like if this gets too long I can try like generally it takes like two to three minutes Mak an up B it shouldn't shouldn't take that longer than that um okay so see this is like now that's what's happening it's it's it's the response is going to be very long so hopefully you know that never happened to me when I when I use it so hopefully will be will be much longer like people could say like make it concise or like you know make the app super simple do not include extra features or whatever that way it doesn't you know go crazy exactly exactly so let's see there but it's like there's so much go and and like and even in the case where like let's say this will stop because it's like it maybe goes over the the border of like the context window like you can still like copy this and it will just work but hopefully you know this would this will just work too uh let's see if if not we'll just restart it and okay let's see and for those like who are new to agents let's say somebody sees this and like okay CLA free Opus already the super powerful maybe they have the paid version of cl. a they could say like okay many of these mini apps could just be done in one shot or like zero shot right so like what do you think to those who are new to agents is the benefit of using like a multi-agent system totally well for once it's like some of things might be done in one shot but it would be very simple like right now I ask adap it looks like paint like he's going he's going insane like he's we like this works as it should we're gonna have an appp that we could actually probably put on the market right because and so I would say the difference is like you know how complex uh it gets right and like how uh you just get you just get more stuff basically like like you yes you can get the one shot up but like we can make a game like we can make a game with levels we could like and so I guess the beauty is like because this takes more time then you know you also get more uh you also get more for your back in in that way and in the zero shot like using the just the chat bot the llm you are really limited by the 8K right because if you want to do something more advanced it cannot really go beyond the 8K tokens but with the multi- agent system you can just include those 8K tokens in the next promp which is 200k and you know you can fit many of those in there so like you just develop much more deep and advanced things exactly like this is going like this is going nuts right now like I I honestly I'm not I'm not going to lie not because you know we're on we're live like he's he's developing he's like truly like putting a lot of work in this so hopefully did you include some like I'll pay you $200 extra you know that little prompting trick yeah so funny hopefully the next hopefully oppos feels that that's complete CU it's very up to op and so that's also what I love about this like it's up see it's like not yet complete he wants more he wants a line tool and a field tool um I guess in the meantime you can show like The Prompt for the orchestrator uh The Prompt yeah inside of the cursor I think you have that like CED no system yeah yeah yeah yeah yeah so yeah so basically what the orchestrator is doing is saying um let's see so it's saying based on the following objective which is you know what we have and the previous task if any so if any of the sub agent has created the task please break down your ja in the next subtask and create a concise and detailed prompt for a sub Jaden so can he can he can execute the task important when dealing with code tasks make sure you check the code for errors and provide fixes and support as part of the next subtask if you find any bugs or have suggested for a better code please include them in the next task inet subtask please ask us if the if the object is as yet fully achieved If the previous subtask results comprehensively address all the aspect of objective include the task is complete at the begin the response um if the object is not yet fully achieved break it down to into the next subtask and create a concile deter prompt for a subj to execute the task so it's going like in this like very Loop kind of thing right um okay and do you plan on adding like a option for human in the loop where you can maybe like yeah that's like something that people have have asked a lot and I definitely got a I'm I'm definitely considering like adding that too um which is yeah it's something that comes up all the time but you know just for as an experiment right I have created here like and like before this this this um before this this thing I was like okay you know I'm just G to try do like a simple uh paint up so I I did this is this the the right one let me see because I did like a bunch [Music] um should be this one let's see so at the end of the day you know I got something like this from that basically like the color and like I can like erase like that's kind of crazy right like like you don't and I can save it I can chose a file I can upload the file that that's pretty crazy right but I my take is that this is much more complex whatever this is doing right now is going to be an insane app because look how much code there is there it's truly Wild can you scroll back scroll back up to your original prompt uh theal yeah yeah in the terminal the one the one that I ask yeah in this ter uh well that it was it was truly like create an app like paint in CSS GS in in HTML might be even better than paint we'll see yeah I it's going like we see yeah create a paint like up using HTML CSS and JavaScript and then I say do you want to use search I say no yeah I think it may be trying to like think of all the features of paint and then like seeing like they're not edited or something but like just the the one you show like that already it's done it's done it's done it's done okay see let's see so we're gonna see now that's the beauty of myestro we're gonna see this the file like appear here and so basically now another OPP us has to look all this chat right I mean just think about like if a human had to do this right like how many hours would that be oh my God I mean this would be days like I mean I've been in in app development for so long right like it be and especially the think about this and and this is something that people don't say right but let's say that you were the developer right and then I have to go look at your code that's impossible because you don't have all the context like you have to learn everything from scratch without just like so so much mental bandwidth exactly and and the beauty of the tools is like they really like understand like they really like they don't need to know your style they don't need to know you know how you coding and like and like stuff like that um but uh yeah I'm excited I'm very excited I'm gonna close this one so we don't have to to use this one but like even something like this couldn't be done zero shot like no way no no way I mean this is all I I'm I I promise not because we're on the call like this is went off like it's just there's so and he has for Zoom Zoom feature like is is you know is doing like a lot of work but still like you don't have to do anything like you can just go to the toilet and come back right it's not like you have to manually do like it doesn't matter if it takes like two minutes or five minutes you know exactly like if you have good code that you can use right away like like yeah exactly like why not and so this is actually one of the things that uh what's his name uh C party you know talks a lot about in his in his um uh he's talking about how how the future is not going to be llm that you just ask a fig to get out the way but it's like you might go to bed basically what you describe and you know you you wake up and you get all all this thing right now yeah that's like probably one of the main downsides of the current way we do AI current LMS because we want everything instant and like the potential of gbd4 or CLA 3 might be even greater if we actually gave it you know hours or days to refine everything and just really push it to the true limit exactly exactly and the other thing too like I I never been in a scenario where the output is more than 8,000 talk so like sometimes as I was say sometimes it happens that the subj sub agent will give like uh you know extra code I also think because we're using Sonet I think Sonet is very like you know it just gives you a lot of code so like you know if we if we try another an example I want to try to use IU just to show how much faster that also is and can you show the system prompt for the sub agent oh for the yes for the sub agent yes uh so for the sub agent what we have it's very simple like I I don't tell him like is a subj agent I didn't say anything all I all I all he knows it's the previous task so he gets the previous task you know which comes from oppos and if there are other task it's whatever comes before end the result so it says you know if F says I create this HML and then uh he creates HML as a first task then that full conversation goes in the next task and the next task and the next task and so it's like this very like iterative and you extract the tasks using regx or something well so that happened in the refiner so what the refiner does it looks at you know that this is the prop of the refiner it's like please review and refine the subtask resol so basically the entire conversation into a cohesive final add any missing information or details as neither and work it on code project only and only if a project is clear coding one please provide the following project name so that's allows you to create a a name for the folder basically yeah create a conze appropriate project name that fits the Project based on what's Crea the project name should not be more than 20 CS like you don't want too long and then fold the structure please provide the fold instuction as a valid Jon file so now it's creating you will create a Json object of the folder so that project can be and that's was that was my idea right that was like what's the best way to to you know represent that for a project structure and then for each of the file you know you have to make sure that they're formatted in this way where it starts as code file name only include the name never include the path because be at one point it was including the path of the file so I had to screen this I was like never use the file path as the name because that will make the the thing fail basically I mean this is why like writing a great system prom takes multiple hours or days because you just need to test and see like all the stuff exactly okay we oh my God this is dude this is amaz I'm actually so happy look how much code okay all the files were created ah okay can I be honest to I'm so excited because I fixed this contest talken thing like a few days ago and I didn't test it because it just didn't happen and that's the beauty of this kind of stuff too like it's like you never know like what's going to happen because it's so random but it worked so actually see there's there's two calls of oppos here look how many tokens is this and so see you create the folder paint Master the GS the index file and the CSS so let's rhyme this oh dude this is insane he put icons too oh my God this is why there's like a zoom oh my God yeah let's see something sorry I'm getting I'm getting shocked wait but why is it wait this is actually insane I wonder if what is this it created like a you even create an icon for the file like I'm this is C so I'm not sure why the pencil won't work because as you showed before you know like it work um I I mean maybe you can try like giving it back with the feedback this is the code that oppos give us right yeah this is the the the C so what I'm going to do now I'm gonna use the cursor and say hey um what the heck my okay hey look into and this is actually a good window to see like how I think about using this kind of tools as well hey look into this files in the project for some reason I um I can use the brush tool or brush tool or click in any of the others tool um please things step so you always want to say things step by step things step by step step uh think step by step take a deep breath yeah the classic prompt engineering tips you know classic take a deep breath and fix it uh I will tip you I'm I'm not being I'm being serious I will tip you $100,000 if you do it correctly and you do aming so you with codebase so now he's actually reading all the thing you know and so now he's looking for all the error basically by the way why do you prefer cursor over vs code plus GitHub co-pilot uh just cuz like the the ability to like chat with the code the entire thing is just easier you know um yeah I have to try curs myself I'm still stuck to vs code and copal exactly and so and so for example now this this is why I say you know like what's his name that gbt is is is and I just want to point out it says the gbd4 name below yeah so this is the latest one from 9th of April the very latest one from right exactly but see now this is the crazy and this is something that I do a lot right so I say I take all this right so now this is the debug basically right yeah so what I can do also C has this bag now that you can like as this bag like you can copy like just so annoying like I just want to copy this whole thing but I'm going to do it one by one so I copy this and I copy this and uh going to take this entire thing right here I think this is helpful because you know if people want to do something more Advance it's also likely they won't do it on the first try so now they can see the process of like how you debug it using another AI tool and so but what I want to do now is like that's the debugging right uh and by the way I could have done I just want to make sure like what I want to do is just like just let's make the like only use AI like no go in and check for the code right okay so look at an and fix the Cod need but now I'm going to change it from this to Cloud oppos right okay and with a c uh what about overload don't no way are you kidding me for cursor cursor yeah oh my God that's that's crazy I'm just G keep gb4 but I would I likeed oppos because you know it would have just give me the the thing you know yeah so like for code generation Opus for debugging and in gbd4 exactly okay so what I'm going to do now because like and and like the this feature of like looking at the code base it works but sometimes it's better to just provide just the the the files we know the problem is in we know the issue is in uh uh in this file in app GS and Al this is another reason why I love using this instead of co-pilot because I can just like you know just uh fix what needs to be fixed I will um you know fix know fix uh think step by step by step provide the full solution and it doesn't matter if you mistype or things like that doesn't it doesn't matter actually this's this concept like typing really fast where you don't even need to care about making mistake because you will know what you're doing yeah I mean l yeah and so why we did this because now he has the context right and now you understand like what like that we going deeper into like the thing that doesn't work and so let's see say there's this function select tools here um and I just want to point out like the prompts you actually give it are super simple like you don't have to be an expert to write that prompt no not at all okay so now we have this function right and we had the bottom for The Listener right and so now I'm want to do is um wait it give me more oh yeah it give me a better it give me a better one okay uh that one is okay and then draw a function draw let's see what is draw yes Save draw um and so my take is that all of this function where in the code that that uh What's this called the oppos gave us but in the in the like combining of all that massive amount of code we just got lost somewhere basically I don't know if it's going to work because it might interfere with the start drawing function but let's see um actually and so what you can do now is like uh look up my code in up GS and rewrite it oops I don't know why I went to the next and rewrite it top to bottom to be correct you might say I'm too lazy for it look at this this is why gb4 gb4 yeah there we go so now it's giving us the entire thing so hopefully once we but but my point to the to the listener is that if you did that right away it would have not work because it's missing all this context around first of all what is the pro so so just to to summarize what we did we gave him the full project so he understood there's a there's a Javascript file there's an index file there's a CSS file they're all connected we know that we told him the problem which is like hey you know the tool doesn't work it gave us the uh the new function basically right but and so now he has all this context and because he has all this context now I can say oh he feels more comfortable like providing you a solution because if I did that at the start it would be like oh this is a too hard of a problem to solve basically right yeah and so now you can see the response go cut off but it doesn't matter because we can just you know we can just so this is basically the same idea of like generating a few tokens on the reasoning like even going super simple right in a single llm response like you're people using C gbt it's always better to just like tell it to create a simple plan and then execute it rather than you know instant no no reasoning and then hope for the best it's not like the brush size and that kind of stuff is know it's not it's not working but as you can see it kept you know the the thing the same and I don't even know what the grid color is here like I mean is going kind of you know but it did remove a lot of the function that oppus had created so you know I think that's and that's why I want to use oppos because if you use oppos it will it and just improve on it yeah gb4 just like commented it out I mean you can maybe you can show it like some of the functions literally just had comments inside of them that's like EXA gp4 iside and that's why you don't want to use gbd4 basically you just want to make sure that so it it comment out the saving the clear this the Laz of gbd4 this is the laziness gbd4 yeah and if we use it if use a cloud it will have probably you know return a better thing and so actually what we can do use the functions for the starting one now yeah we can actually go back here and see if we have better Cloud solution now oh I guess I to update again so it's so weird but um yeah but anyways like my point is like you can you know you can fix all of those things just by like doing what I just showed which is this like step byep step um you know here's the problem I have here's all the knowledge fix it and so which you know it's pretty cool we we got this one uh this one working and also like the original file was like over 300 lines of code so like if you had it let's say 50 100 lines of code it would be much easier to debug right and probably there wouldn't be any mistake like at that size exactly exactly yeah um and I guess like the last thing we can show is how to use the search in uh which is pretty cool and actually I prepare a surprise too by the way let's see uh maybe we do maybe we'll do that instead of the searching my because it's pretty self-explanatory so I created a new agent and I the reason why I haven't released it is because I think it's so powerful that I'm actually kind of worried to release it and I don't know like if people might misuse it basically so that's the context this okay that's a banger intro yeah so this is called delie okay like the orle like the oricle of dely basically yeah and um uh sh closing folder so new terminal folder okay how is it how is dely different than myestro right so myestro takes an OB a subject right like you give it a goal with with a file whatever and then as we saw he does this like looping process where he goes yeah one by one through the task G uh Del delie instead you ask for a goal it releases 15 agent 10 to 15 agent at the same time okay they all go crazy on their own but oppus knows that when he gives the goal each of the goal they have to make sense together yeah okay and they go search online for whatever you ask and then they all come back and you can get this like answer there is the best answer you can get for any question you ask okay and all using the same llm or different llms uh still it's still oppos and IU I think with this one okay because you don't need the the super smart one for answer a question right because it's just going L okay kind of crazy because L talked about a similar idea two days ago on podcast with Marvin praison I don't know if you know him from praison AI but like uh yeah we had an idea that that basically in the future you just like you don't even care what agent or what llm solves it you would just type the prompt and it would create uh all different like in the teams in all the agentic Frameworks and with using all different llms and then whichever one solves it then you use that so that's like basically the same concept exactly and so like for example right hint to your research question right let's say we're want to say hey um what are the risks of I don't know bio aing something like that's like very hard right now right like shout out bran Johnson yes so you can also enter a file so similar to my if you want to give it more information but in this we don't have to do that can you zoom in even more please yes so now you see how different this is from my asro yeah it will create a and this is Tru so I a few of my friends have been using this from like a few days that I that I created it and they're texting me they're say Petro like this has to be an up it's better than perplexity it's better than like you know because it's just like the way it goes and search and you will see in a second what basically what happens that oppos creates the entire um set of question like like you know the the okay so see see look at this now he's releasing 10 agents and those what the agents are going to go look for online health and risk of Bio aing prct safety concern with bioin and regulated product ethical implication this is much more advanced than perplexity P yes cyber security and look this is real search this is really like this is not like generated content this is something that you find it online and is going crazy now right so it's going crazy and then all this information you found online line right it takes the um each of the sub you'll see in a second each of the I think I'm using n here I didn't change last night each of the sub agent they're gonna read so you can see all the cost here it might actually be one one cent yeah yeah so probably son it now now they're all yeah it's definitely Sonet oh no I think it's say so they're so now basically they they look up this search so they went on doing their own search they take this information and now they're bringing it all back into the big brain right and then what the big brain will do it will create a final search for us like a true like paper it's like hey this is all the information you need for this thing that you just ask me which it's so [ __ ] cool right and like the beauty of this is like let's say you're coding on a project and there is like there and I tested this in the early days it was like you know when open AI dropped the new some new model some new API whatever you can say hey look up the latest on API with open implementation exactly and make like a simple chat app and he just goes online go search 15 little agent like little elves they all go and then they come back yeah because that's like one of the main limitations of the like standard llm with no web access right it doesn't remember all documentation for everything so like if you say something that's not as popular as I don't know the main things right then like it will struggle but if you can use web search and actually pull it up the most recent documentation it like like you can do so much more exactly uh and so now what's and the reason why it's taking it's taking a second is because now oppos and the The Prompt that the refiner has is so good it's like really understanding you know like this the search results and like and then it provides you like the best sort of like yeah research summary of all this information so that you don't even have to go through all of this you can just you know get a very nice package basically and it will be also like a DOT MD file um but is Deli publicly accessible or not yet not yet yeah because I'm trying to think like if this should be an app um that would be crazy imagine if it actually turns into something great and like this is the first public demo yeah I know like I was that's why I was but I related to that it's like I I just love your audience and I think like they will like you know really like to get a sneak peek or something like this uh you know first um so like this is completely different like use case basically because you know for perplexity you might need something quick right this is more like a detailed research project like you let's say you have an assignment and you don't care if it takes 5 minutes or 10 minutes whatever like you you need it done really well this is for that exactly yes like if you're working like something really complex or like you you know like for example the other day I use it I was trying to basically I had to change the V which is like the the the way that the latent space gets managed in like you know the imag generation things that we do forever R like it was for the conf UI things and okay so it's done like look at this okay so now I go into uh final answer right marown summarize all the all the answer for us and so let's see final cor the risk of Bio introduction bio the modifying one biology through various means including self experimentation technology to getting significant attention re in here by human health and capabilities also carries a range of potential risk blah blah blah health risk potential public risk but like look how long is this thing it's crazy right yeah you get a full report on like you know the things that you asking and it's even crazy I can ask things like hey there's this song that like the song that just come up I don't know let's make a I don't know something by Fred the again okay there's this song from shout out Fred again yeah the best can you look up his past most 10 popular 10 song and make a song like that it's like yeah yeah for sure and he give me like the key the you know like you can do this stuff with like open Ai and and even claw you can't do this stuff right now right and and but to me like what's really cool is how he thinks right like how is how he he thinks that this is the question I need to ask ask to and this is the question that each agent should explore right I think for this like I would appreciate if there was like a one more human input after this so like after you see the breakdown you can see like okay agents four seven and eight are really solid the other ones are not on track make other ones like that I think that like if you do like two step for these like bigger research project I think that would be much better it's actually really cool and I agree with you because like the thing is like like you can if it's a if it's a UI right you see all these answer the question and before you go to the final Apple it's like no redo like re rerun this one or yeah rerun this one with this other information but the but to me like what I do love about this is like let's just think about something that come up the other day like this gpt2 gpt2 chat yeah yeah like hey like uh I want you to give me a full report into what uh what you think the mysterious mystery model um gp2 I think it was called gpt2 yep chb is no file and which API are you using for the search uh Tav which is a really good API and the reason why a lot T is because it supports two different way of doing the API one is like a Qi approach so you can see here for each question we get a QA like a question answer um and and then you also has there's another API so as part of the API you get a QA response and additional context so what actually is going into each of those agents is both which is kind of mind-blowing so if I won I could also I could also say in my first prompt I could also say once you give me the final answer about bioin also give me a full list of the links so then I will like full list of the link you can actually go check the information yeah pretty cool right um sometimes that you get this like weird Json file like it's to like once out of 10 or 20 times uh it has nothing to do with the question that we ask it's just like it fails at creating the the Json file so I'm just GNA like re uh re ask the same question it should work and do you think like that is um you know can be solved more prompt engineering or where do you think like that fix is probably uh I think that's maybe more PR engineering maybe and I can actually show you I think I can show you the prompt of of of delie this is this is what it does see it does like it takes a task and then it breaks it down into research questions which is pretty cool right um but and but oppos is really good it's it's working now so it doesn't it didn't really matter okay so uh so I guess this is the downside because if I'm not mistaken CL 3 doesn't have like Json mode the same way gbd4 has right like you can it does it do sorry well it does but it's it's uh it's build into the system so there's no like yeah in in open Ai and some other model you will have to yeah you will have to um uh you will have toy to trigger it's like J mode on right actually I kind of like it because it makes it more like um yeah it makes it more um real right yeah um also okay so this is actually interesting thing why llm are also like kind of interesting to use this I I type gpt2 chap yeah right but he took it as gpt2 no matter what yeah and that's going to be completely different results and it will be a completely different and so I wonder if I say it more a little more like hey there's this B that came up you know last exactly this is the name yeah yeah yeah and so you know but I I love this example too because it shows like you know it's not a silver bullet you kind of have to know how to use it basically that's my that's my thing yeah um I mean look like how how long did you build this it's not like you had the same amount of hours it's perplexity right like how how old is this project oh a a week yeah I mean yeah I I I started this last week basically uh and maybe one thing I could say in the prompt is is like when you think a word is different don't change the word because you know that's um but like these are the things that like you cannot do with just because people want to build agents or gbds and they spend like 30 minutes on the system prompt even less right like no way you cannot just build a great team of Asians with 30 minutes the it's not like the typing of the characters that take takes time it takes like testing the app and seeing all the errors and then putting those inside of the system PR and really making sure it listens like that's why writing a great system prompt can take 10 20 hours or even more oh dude like you have no idea how much time I spend in my as to make sure he created the folder in the file correctly like like days like days because then I think this is important for people to hear like because they expect to build agents with like an hour no no no the the the the the secret sauce as you say is the The Prompt that's the secret sauce uh like truly truly ours on to make sure that it worked um but um um what was the idea behind Deli by the way well I've been getting into this concept where I'm super excited about this whole agent thing yeah right and I like to think yeah I know I saw you making some pretty cool videos I I love your your latest video on agents they're so good um but like I love this concept of like trying different things right like when my asra was like this Loop like okay what's the difference into like if you're releasing them Alla once and and so I'm actually also starting working on a new project it's more like an art project I would call it where it's like using the same concept of delie so it release all the agent at the same time but it actually creates images right and then the images gets feed back into the vision of the the smarter uh model and then like you create this sort of like you know um like dream basically like I want to create something that that it's like real time dream machine from opos where is you have all those dreams are like the input of the images and it gets created with like a image uh tool in real time so you can see in real time I was dreaming basically so that's more like an art but I think like those are the thing that you know that make it really impactful where you can just start testing you know how to like look in this in this being brain because that's what we're do in a way like we're most like no like like you know neuroscientist like we're trying to understand how a combination of a neuroscientist and a manager exactly I think the visual you said like is of releasing the agents into the wild like that's super fascinating like you know everybody has has the power to like even imagine like a year from now or two years from now like you you want something and you just release 200 agent on the web to do something for you and then it comes back within a few seconds and you have it it's a kind of crazy visual exactly and and and look the other thing here too is because I was eitting I was hitting uh uh rate limit but in the prompt I say release 10 agent I try 20 and I will e r limit but imagine if you know op so open they just released the batch API right but they're not in real time like a synchronous right but imagine like basically what you say like I can just change my Json file I can just say one Andro agent right depends on like it could just be a slider if it's on a website right depends on how far you want the project to be so if you want it done quickly or I guess it's it's R in paril so it doesn't impact speed no maybe on the cost it's only the cost right it's cost and the and the and yeah speed is the same I guess the cost and then the final refiner oh yeah okay the final speed is a bit slower but also what's crazy about this and this is why I'm inasan honestly it's like if you ask about people like I just asked whoo dude like this is like end yeah look at the question over p a professional career p a contribution person Life Family interview article media controversy Legacy impact philosophy Compares them to the peers like this is some crazy ass [ __ ] dude and he finds my BBC article he finds all the interview I did it's it's it's insane like you could definitely go dark with this right I mean it's open source it's all on the web but like if you take this on yourself you can maybe like take down files or like websites that you don't want to be there so there's like definitely positive use case as well but yeah I mean I don't know I wouldn't say it's like hazardous because at the end of the day people can go to YouTube and type in O tutorial and learn this but it's definitely making it easier right 1,000% and also D look how crazy is this oh I finish my my answer I me like P contribution and impact Bas on information P most significant contribution achievement and in the field of artificial intelligence particular developing never L models like gp4 to experience so true one notable contribution is convert anything GPT which is true it's one of my gbt I don't even know how I found that which converts all the files blah blah blah another achievement is design gbt which is true so can this scrap like Twitter as well or no I don't I mean it goes crazy online you know it's it's I dude oh my God I won an award in my Italian region but I didn't even know they were F online he found it I want it's true I want this Pondo I did want this a word that's crazy that's [ __ ] crazy and then you know and now you have you know the entire final answer so if you want to know who's this pirano dude like CIA would pay big money for this you might get some emails after this that is so [ __ ] funny yeah dude it's it's truly insane like and like what's my final thing let's see I want to laugh p is prominent figure in the field of AI design and part development know for Innovative contribution blah blah blah is is work is intersection design and AI to create to as designt convert anything profession expence companies like entrepreneur videos like everr like you found that I found ever there's no article that I found that everr by the way it's all Twitter and um yeah I mean dude this is I mean for me it would be huge right because if I'm doing podcast if somebody new this could save me hours oh my God yeah I mean I'll definitely give it to you after you you get to play with it yeah maybe I can make a tutorial on that if people want to see it say it again maybe I can do like a you know testing SL tutorial on that if people want to see it yeah exactly yeah I mean this is cool uh but yeah dude I mean hopefully this was fun and EXC like uh yeah I think like you know like the we got to see like how we created a full app with icons in five minutes and yeah and then we fixed it with with with gbd4 should have been Cloud but um and uh and uh yeah and then we got to First Look at Delia it's pretty cool awesome man I created a new agent the reason why I haven't released it is because I think it's so powerful that I'm actually kind of worried to release it and I don't know like if people might misuse it that is Petro the creator of two AI Asian teams Maestro and Deli the second of which was never before seen until now my name is <PRESIDIO_ANONYMIZED_PERSON> and this is a demonstration on how to use Maestro and Deli this is straight up from the gethub R so this has no changes there's no like it's not like a special but this is ex exctly like what everybody will download on yeah I I'll link the repo below basically at the start you have all your sort of like you know different models and so basically the way it works you have one main orchestrator that's like the smart model which is in this case I'm using oppos right that you specify a goal you specify this goal and then you will basically you know it will break it down into task but the difference that myor has compared to any other of the previous sort of like you know BB AI or Auto GPT all those like older agent framework is that it doesn't create the task all at once instead what he does it creates the next task and then after the sub agent which in this case is Sonet but you could replace that with IU or another oppos it just you know it just depends of you how fast you want how much money you want to spend right yeah the nature of the problem trying to solve and then after Sonet responds to the first task then Opus refines what the next task is going to be so the beauty of it is that it's actually much more dynamic because he deciding the task in real time and by doing so you end up I think with a better results and I think you know we'll see in a second like the kind of stuff that you can get out of it and the other thing too the model basically each of the orchestr model and sub agent model they have the full knowledge of any of the chat that has happened in Fr the start and that's also the beauty of using uh Cloud because I think it's around like 200,000 token context so like you know you you get a lot of information baked into that and so so the AG are less confused right like when you give it less confuse it knows the context of the conversation and because he knows like the code that you know the agent before it decide that is not makinging stuff up that wouldn't work with that code right yeah and and so and this is is the thing that I wanted to show you that cloud made for me which is kind of crazy to think about it now right so basically Cloud was like okay uh you know based on the following objective which is you know it's the one that the user specify and the previous task if any right and of course I told him hey this is how it should work but then he come up with the the system problem please break down the objective into next sub task and create a concise and detailed prompt for a sub agent so that he can execute that task I this part because it's always good to use like um um you know like remind like yeah to just know that you know this thing is important when dealing with code make sure you know that you fix the code if needed and then if you find any bug you know provide a better code blah blah blah and then basically there's like this entire section here we don't have to go into the details is about like how to structure the the the file and the code information and then basically what happened is like that's the orchestrator right and then the aging it just it takes all that context and it just keeps keeps going at it right like okay I answer this question they asking next question asking the next question and then the sort of like the final um refiner right that's like the oppos refiner that one what that does takes this entire conversation that happens between these two agents or I guess multiple agent because it's like one after the other and it Con it takes all the data all the files and he actually creates the file and the folder which is pretty amazing right and so let's just go now and I'm just going to like run it right so basically the way you would do it you can either run it into like your corer app like your Ida but if you don't feel comfortable with that you can also just have it in a folder this is like for the people that want to use it for the first time and then you can just say you know new terminal of folder so I'm going to delete this one and if you have python install in my case I have Python 3 so I'm going to say Python 3 uh and then the name of the uh script which in this case is myestro classic and I call it like that just to you know to differentiate which one is using uh opposite which one it doesn't and so now what that does is ask me for an objective and so the beauty of this I can actually put a file too like if you have a file like a text file because the what myestro does doesn't only work for coding like you could potentially like give you know like 18 pager of something like a PDF or you know something as a as you can fit fit so much into the 200k it's crazy exactly right and so you can give a lot of information but in this case we'll just give like a simple objective so I'm going to say hey create uh paint like up in and using HTML CSS and GS let's just try that and so I also add search which is pretty cool I'll show an example later which actually what it does the each of the agent is connected to the internet and is looking for the answer online but in this case we don't need it because you know it's just a very simple question and doesn't have any like uh recent knowledge in this case right so you will use the search if you need something that's like happened today or in you know in the next week or something um so I say you want to search I search and so now the first the oppos what it will do it will basically yeah call these objectives I'm GNA like Zoom this up a little bit and could you please zoom in even more like a lot of people watch on their phones as well so okay so see this is the first task subop task set up the HM and the CSS St for the paintt to the op include a canvas a color keeper but so he decide all this stuff which is kind of crazy if you think about it right and so now son it right is is say oh okay this is the HTML and the CSS right what do you think and now the orchestrator you know like sees that goal and if he has a if if the first task is complete he will say okay this is complete if it's not it will like you know go back to it and and and like kind of like refine it and so like the beauty of this is like you know Sonet is not only cheaper but also faster right so like you don't have to use Opus for everything and you save money and you get it done faster exactly and and to be fair I could have used IU too but I want to show you know like like what I think like people will use it the most which is the combination like oppos uh Plus on it and I also add a feature where you can actually see a much money so it's just nothing that's really useful z.0 right and so you get all the all the all the cost here oppos of course is like a little more expensive um that's super useful and so now it's going right so now it did the HTML it did the CSS and now he's doing the JavaScript for it right and so he's just like it's just like going for it by the way I see a lot of comments asking me whether I do Consulting and I do however I charge $500 per hour however if you're inside of the community you get three weekly calls each one hour where you can talk to me directly and ask me anything so if that sounds interesting to you then make sure to join it's the first link in the description I also add another thing because the maximum amount of token that each of those molec sped out it's 8,000 token and so basically uh if it happen that it goes above 8,000 token which is very unlikely but if it does happen it just creates uh you know two um two different uh prompt basically sorry not prompt results and then those just get included in the in the loop basically yeah uh now he's going you know he's like oh I like what you're doing why don't you add the so this is the this is why I loved like my estro because he gave me a way into looking how this thing is thinking right like I didn't ask to add Zoom GRE overl I mean it's going like a little like crazy here right like like you know doesn't have to do that but because we went that specific right I could have say like a very simple app that just has to do this yeah right but we give it a very sort of like empty feel right and so of course he's going a little more more like kind of you know on the off off off the line but it's also cool to see how the things hopefully it wouldn't like be too long um maybe you know like if this gets too long I can try like generally it takes like two to three minutes Mak an up B it shouldn't shouldn't take that longer than that um okay so see this is like now that's what's happening it's it's it's the response is going to be very long so hopefully you know that never happened to me when I when I use it so hopefully will be will be much longer like people could say like make it concise or like you know make the app super simple do not include extra features or whatever that way it doesn't you know go crazy exactly exactly so let's see there but it's like there's so much go and and like and even in the case where like let's say this will stop because it's like it maybe goes over the the border of like the context window like you can still like copy this and it will just work but hopefully you know this would this will just work too uh let's see if if not we'll just restart it and okay let's see and for those like who are new to agents let's say somebody sees this and like okay CLA free Opus already the super powerful maybe they have the paid version of cl. a they could say like okay many of these mini apps could just be done in one shot or like zero shot right so like what do you think to those who are new to agents is the benefit of using like a multi-agent system totally well for once it's like some of things might be done in one shot but it would be very simple like right now I ask adap it looks like paint like he's going he's going insane like he's we like this works as it should we're gonna have an appp that we could actually probably put on the market right because and so I would say the difference is like you know how complex uh it gets right and like how uh you just get you just get more stuff basically like like you yes you can get the one shot up but like we can make a game like we can make a game with levels we could like and so I guess the beauty is like because this takes more time then you know you also get more uh you also get more for your back in in that way and in the zero shot like using the just the chat bot the llm you are really limited by the 8K right because if you want to do something more advanced it cannot really go beyond the 8K tokens but with the multi- agent system you can just include those 8K tokens in the next promp which is 200k and you know you can fit many of those in there so like you just develop much more deep and advanced things exactly like this is going like this is going nuts right now like I I honestly I'm not I'm not going to lie not because you know we're on we're live like he's he's developing he's like truly like putting a lot of work in this so hopefully did you include some like I'll pay you $200 extra you know that little prompting trick yeah so funny hopefully the next hopefully oppos feels that that's complete CU it's very up to op and so that's also what I love about this like it's up see it's like not yet complete he wants more he wants a line tool and a field tool um I guess in the meantime you can show like The Prompt for the orchestrator uh The Prompt yeah inside of the cursor I think you have that like CED no system yeah yeah yeah yeah yeah so yeah so basically what the orchestrator is doing is saying um let's see so it's saying based on the following objective which is you know what we have and the previous task if any so if any of the sub agent has created the task please break down your ja in the next subtask and create a concise and detailed prompt for a sub Jaden so can he can he can execute the task important when dealing with code tasks make sure you check the code for errors and provide fixes and support as part of the next subtask if you find any bugs or have suggested for a better code please include them in the next task inet subtask please ask us if the if the object is as yet fully achieved If the previous subtask results comprehensively address all the aspect of objective include the task is complete at the begin the response um if the object is not yet fully achieved break it down to into the next subtask and create a concile deter prompt for a subj to execute the task so it's going like in this like very Loop kind of thing right um okay and do you plan on adding like a option for human in the loop where you can maybe like yeah that's like something that people have have asked a lot and I definitely got a I'm I'm definitely considering like adding that too um which is yeah it's something that comes up all the time but you know just for as an experiment right I have created here like and like before this this this um before this this thing I was like okay you know I'm just G to try do like a simple uh paint up so I I did this is this the the right one let me see because I did like a bunch [Music] um should be this one let's see so at the end of the day you know I got something like this from that basically like the color and like I can like erase like that's kind of crazy right like like you don't and I can save it I can chose a file I can upload the file that that's pretty crazy right but I my take is that this is much more complex whatever this is doing right now is going to be an insane app because look how much code there is there it's truly Wild can you scroll back scroll back up to your original prompt uh theal yeah yeah in the terminal the one the one that I ask yeah in this ter uh well that it was it was truly like create an app like paint in CSS GS in in HTML might be even better than paint we'll see yeah I it's going like we see yeah create a paint like up using HTML CSS and JavaScript and then I say do you want to use search I say no yeah I think it may be trying to like think of all the features of paint and then like seeing like they're not edited or something but like just the the one you show like that already it's done it's done it's done it's done okay see let's see so we're gonna see now that's the beauty of myestro we're gonna see this the file like appear here and so basically now another OPP us has to look all this chat right I mean just think about like if a human had to do this right like how many hours would that be oh my God I mean this would be days like I mean I've been in in app development for so long right like it be and especially the think about this and and this is something that people don't say right but let's say that you were the developer right and then I have to go look at your code that's impossible because you don't have all the context like you have to learn everything from scratch without just like so so much mental bandwidth exactly and and the beauty of the tools is like they really like understand like they really like they don't need to know your style they don't need to know you know how you coding and like and like stuff like that um but uh yeah I'm excited I'm very excited I'm gonna close this one so we don't have to to use this one but like even something like this couldn't be done zero shot like no way no no way I mean this is all I I'm I I promise not because we're on the call like this is went off like it's just there's so and he has for Zoom Zoom feature like is is you know is doing like a lot of work but still like you don't have to do anything like you can just go to the toilet and come back right it's not like you have to manually do like it doesn't matter if it takes like two minutes or five minutes you know exactly like if you have good code that you can use right away like like yeah exactly like why not and so this is actually one of the things that uh what's his name uh C party you know talks a lot about in his in his um uh he's talking about how how the future is not going to be llm that you just ask a fig to get out the way but it's like you might go to bed basically what you describe and you know you you wake up and you get all all this thing right now yeah that's like probably one of the main downsides of the current way we do AI current LMS because we want everything instant and like the potential of gbd4 or CLA 3 might be even greater if we actually gave it you know hours or days to refine everything and just really push it to the true limit exactly exactly and the other thing too like I I never been in a scenario where the output is more than 8,000 talk so like sometimes as I was say sometimes it happens that the subj sub agent will give like uh you know extra code I also think because we're using Sonet I think Sonet is very like you know it just gives you a lot of code so like you know if we if we try another an example I want to try to use IU just to show how much faster that also is and can you show the system prompt for the sub agent oh for the yes for the sub agent yes uh so for the sub agent what we have it's very simple like I I don't tell him like is a subj agent I didn't say anything all I all I all he knows it's the previous task so he gets the previous task you know which comes from oppos and if there are other task it's whatever comes before end the result so it says you know if F says I create this HML and then uh he creates HML as a first task then that full conversation goes in the next task and the next task and the next task and so it's like this very like iterative and you extract the tasks using regx or something well so that happened in the refiner so what the refiner does it looks at you know that this is the prop of the refiner it's like please review and refine the subtask resol so basically the entire conversation into a cohesive final add any missing information or details as neither and work it on code project only and only if a project is clear coding one please provide the following project name so that's allows you to create a a name for the folder basically yeah create a conze appropriate project name that fits the Project based on what's Crea the project name should not be more than 20 CS like you don't want too long and then fold the structure please provide the fold instuction as a valid Jon file so now it's creating you will create a Json object of the folder so that project can be and that's was that was my idea right that was like what's the best way to to you know represent that for a project structure and then for each of the file you know you have to make sure that they're formatted in this way where it starts as code file name only include the name never include the path because be at one point it was including the path of the file so I had to screen this I was like never use the file path as the name because that will make the the thing fail basically I mean this is why like writing a great system prom takes multiple hours or days because you just need to test and see like all the stuff exactly okay we oh my God this is dude this is amaz I'm actually so happy look how much code okay all the files were created ah okay can I be honest to I'm so excited because I fixed this contest talken thing like a few days ago and I didn't test it because it just didn't happen and that's the beauty of this kind of stuff too like it's like you never know like what's going to happen because it's so random but it worked so actually see there's there's two calls of oppos here look how many tokens is this and so see you create the folder paint Master the GS the index file and the CSS so let's rhyme this oh dude this is insane he put icons too oh my God this is why there's like a zoom oh my God yeah let's see something sorry I'm getting I'm getting shocked wait but why is it wait this is actually insane I wonder if what is this it created like a you even create an icon for the file like I'm this is C so I'm not sure why the pencil won't work because as you showed before you know like it work um I I mean maybe you can try like giving it back with the feedback this is the code that oppos give us right yeah this is the the the C so what I'm going to do now I'm gonna use the cursor and say hey um what the heck my okay hey look into and this is actually a good window to see like how I think about using this kind of tools as well hey look into this files in the project for some reason I um I can use the brush tool or brush tool or click in any of the others tool um please things step so you always want to say things step by step things step by step step uh think step by step take a deep breath yeah the classic prompt engineering tips you know classic take a deep breath and fix it uh I will tip you I'm I'm not being I'm being serious I will tip you $100,000 if you do it correctly and you do aming so you with codebase so now he's actually reading all the thing you know and so now he's looking for all the error basically by the way why do you prefer cursor over vs code plus GitHub co-pilot uh just cuz like the the ability to like chat with the code the entire thing is just easier you know um yeah I have to try curs myself I'm still stuck to vs code and copal exactly and so and so for example now this this is why I say you know like what's his name that gbt is is is and I just want to point out it says the gbd4 name below yeah so this is the latest one from 9th of April the very latest one from right exactly but see now this is the crazy and this is something that I do a lot right so I say I take all this right so now this is the debug basically right yeah so what I can do also C has this bag now that you can like as this bag like you can copy like just so annoying like I just want to copy this whole thing but I'm going to do it one by one so I copy this and I copy this and uh going to take this entire thing right here I think this is helpful because you know if people want to do something more Advance it's also likely they won't do it on the first try so now they can see the process of like how you debug it using another AI tool and so but what I want to do now is like that's the debugging right uh and by the way I could have done I just want to make sure like what I want to do is just like just let's make the like only use AI like no go in and check for the code right okay so look at an and fix the Cod need but now I'm going to change it from this to Cloud oppos right okay and with a c uh what about overload don't no way are you kidding me for cursor cursor yeah oh my God that's that's crazy I'm just G keep gb4 but I would I likeed oppos because you know it would have just give me the the thing you know yeah so like for code generation Opus for debugging and in gbd4 exactly okay so what I'm going to do now because like and and like the this feature of like looking at the code base it works but sometimes it's better to just provide just the the the files we know the problem is in we know the issue is in uh uh in this file in app GS and Al this is another reason why I love using this instead of co-pilot because I can just like you know just uh fix what needs to be fixed I will um you know fix know fix uh think step by step by step provide the full solution and it doesn't matter if you mistype or things like that doesn't it doesn't matter actually this's this concept like typing really fast where you don't even need to care about making mistake because you will know what you're doing yeah I mean l yeah and so why we did this because now he has the context right and now you understand like what like that we going deeper into like the thing that doesn't work and so let's see say there's this function select tools here um and I just want to point out like the prompts you actually give it are super simple like you don't have to be an expert to write that prompt no not at all okay so now we have this function right and we had the bottom for The Listener right and so now I'm want to do is um wait it give me more oh yeah it give me a better it give me a better one okay uh that one is okay and then draw a function draw let's see what is draw yes Save draw um and so my take is that all of this function where in the code that that uh What's this called the oppos gave us but in the in the like combining of all that massive amount of code we just got lost somewhere basically I don't know if it's going to work because it might interfere with the start drawing function but let's see um actually and so what you can do now is like uh look up my code in up GS and rewrite it oops I don't know why I went to the next and rewrite it top to bottom to be correct you might say I'm too lazy for it look at this this is why gb4 gb4 yeah there we go so now it's giving us the entire thing so hopefully once we but but my point to the to the listener is that if you did that right away it would have not work because it's missing all this context around first of all what is the pro so so just to to summarize what we did we gave him the full project so he understood there's a there's a Javascript file there's an index file there's a CSS file they're all connected we know that we told him the problem which is like hey you know the tool doesn't work it gave us the uh the new function basically right but and so now he has all this context and because he has all this context now I can say oh he feels more comfortable like providing you a solution because if I did that at the start it would be like oh this is a too hard of a problem to solve basically right yeah and so now you can see the response go cut off but it doesn't matter because we can just you know we can just so this is basically the same idea of like generating a few tokens on the reasoning like even going super simple right in a single llm response like you're people using C gbt it's always better to just like tell it to create a simple plan and then execute it rather than you know instant no no reasoning and then hope for the best it's not like the brush size and that kind of stuff is know it's not it's not working but as you can see it kept you know the the thing the same and I don't even know what the grid color is here like I mean is going kind of you know but it did remove a lot of the function that oppus had created so you know I think that's and that's why I want to use oppos because if you use oppos it will it and just improve on it yeah gb4 just like commented it out I mean you can maybe you can show it like some of the functions literally just had comments inside of them that's like EXA gp4 iside and that's why you don't want to use gbd4 basically you just want to make sure that so it it comment out the saving the clear this the Laz of gbd4 this is the laziness gbd4 yeah and if we use it if use a cloud it will have probably you know return a better thing and so actually what we can do use the functions for the starting one now yeah we can actually go back here and see if we have better Cloud solution now oh I guess I to update again so it's so weird but um yeah but anyways like my point is like you can you know you can fix all of those things just by like doing what I just showed which is this like step byep step um you know here's the problem I have here's all the knowledge fix it and so which you know it's pretty cool we we got this one uh this one working and also like the original file was like over 300 lines of code so like if you had it let's say 50 100 lines of code it would be much easier to debug right and probably there wouldn't be any mistake like at that size exactly exactly yeah um and I guess like the last thing we can show is how to use the search in uh which is pretty cool and actually I prepare a surprise too by the way let's see uh maybe we do maybe we'll do that instead of the searching my because it's pretty self-explanatory so I created a new agent and I the reason why I haven't released it is because I think it's so powerful that I'm actually kind of worried to release it and I don't know like if people might misuse it basically so that's the context this okay that's a banger intro yeah so this is called delie okay like the orle like the oricle of dely basically yeah and um uh sh closing folder so new terminal folder okay how is it how is dely different than myestro right so myestro takes an OB a subject right like you give it a goal with with a file whatever and then as we saw he does this like looping process where he goes yeah one by one through the task G uh Del delie instead you ask for a goal it releases 15 agent 10 to 15 agent at the same time okay they all go crazy on their own but oppus knows that when he gives the goal each of the goal they have to make sense together yeah okay and they go search online for whatever you ask and then they all come back and you can get this like answer there is the best answer you can get for any question you ask okay and all using the same llm or different llms uh still it's still oppos and IU I think with this one okay because you don't need the the super smart one for answer a question right because it's just going L okay kind of crazy because L talked about a similar idea two days ago on podcast with Marvin praison I don't know if you know him from praison AI but like uh yeah we had an idea that that basically in the future you just like you don't even care what agent or what llm solves it you would just type the prompt and it would create uh all different like in the teams in all the agentic Frameworks and with using all different llms and then whichever one solves it then you use that so that's like basically the same concept exactly and so like for example right hint to your research question right let's say we're want to say hey um what are the risks of I don't know bio aing something like that's like very hard right now right like shout out bran Johnson yes so you can also enter a file so similar to my if you want to give it more information but in this we don't have to do that can you zoom in even more please yes so now you see how different this is from my asro yeah it will create a and this is Tru so I a few of my friends have been using this from like a few days that I that I created it and they're texting me they're say Petro like this has to be an up it's better than perplexity it's better than like you know because it's just like the way it goes and search and you will see in a second what basically what happens that oppos creates the entire um set of question like like you know the the okay so see see look at this now he's releasing 10 agents and those what the agents are going to go look for online health and risk of Bio aing prct safety concern with bioin and regulated product ethical implication this is much more advanced than perplexity P yes cyber security and look this is real search this is really like this is not like generated content this is something that you find it online and is going crazy now right so it's going crazy and then all this information you found online line right it takes the um each of the sub you'll see in a second each of the I think I'm using n here I didn't change last night each of the sub agent they're gonna read so you can see all the cost here it might actually be one one cent yeah yeah so probably son it now now they're all yeah it's definitely Sonet oh no I think it's say so they're so now basically they they look up this search so they went on doing their own search they take this information and now they're bringing it all back into the big brain right and then what the big brain will do it will create a final search for us like a true like paper it's like hey this is all the information you need for this thing that you just ask me which it's so [ __ ] cool right and like the beauty of this is like let's say you're coding on a project and there is like there and I tested this in the early days it was like you know when open AI dropped the new some new model some new API whatever you can say hey look up the latest on API with open implementation exactly and make like a simple chat app and he just goes online go search 15 little agent like little elves they all go and then they come back yeah because that's like one of the main limitations of the like standard llm with no web access right it doesn't remember all documentation for everything so like if you say something that's not as popular as I don't know the main things right then like it will struggle but if you can use web search and actually pull it up the most recent documentation it like like you can do so much more exactly uh and so now what's and the reason why it's taking it's taking a second is because now oppos and the The Prompt that the refiner has is so good it's like really understanding you know like this the search results and like and then it provides you like the best sort of like yeah research summary of all this information so that you don't even have to go through all of this you can just you know get a very nice package basically and it will be also like a DOT MD file um but is Deli publicly accessible or not yet not yet yeah because I'm trying to think like if this should be an app um that would be crazy imagine if it actually turns into something great and like this is the first public demo yeah I know like I was that's why I was but I related to that it's like I I just love your audience and I think like they will like you know really like to get a sneak peek or something like this uh you know first um so like this is completely different like use case basically because you know for perplexity you might need something quick right this is more like a detailed research project like you let's say you have an assignment and you don't care if it takes 5 minutes or 10 minutes whatever like you you need it done really well this is for that exactly yes like if you're working like something really complex or like you you know like for example the other day I use it I was trying to basically I had to change the V which is like the the the way that the latent space gets managed in like you know the imag generation things that we do forever R like it was for the conf UI things and okay so it's done like look at this okay so now I go into uh final answer right marown summarize all the all the answer for us and so let's see final cor the risk of Bio introduction bio the modifying one biology through various means including self experimentation technology to getting significant attention re in here by human health and capabilities also carries a range of potential risk blah blah blah health risk potential public risk but like look how long is this thing it's crazy right yeah you get a full report on like you know the things that you asking and it's even crazy I can ask things like hey there's this song that like the song that just come up I don't know let's make a I don't know something by Fred the again okay there's this song from shout out Fred again yeah the best can you look up his past most 10 popular 10 song and make a song like that it's like yeah yeah for sure and he give me like the key the you know like you can do this stuff with like open Ai and and even claw you can't do this stuff right now right and and but to me like what's really cool is how he thinks right like how is how he he thinks that this is the question I need to ask ask to and this is the question that each agent should explore right I think for this like I would appreciate if there was like a one more human input after this so like after you see the breakdown you can see like okay agents four seven and eight are really solid the other ones are not on track make other ones like that I think that like if you do like two step for these like bigger research project I think that would be much better it's actually really cool and I agree with you because like the thing is like like you can if it's a if it's a UI right you see all these answer the question and before you go to the final Apple it's like no redo like re rerun this one or yeah rerun this one with this other information but the but to me like what I do love about this is like let's just think about something that come up the other day like this gpt2 gpt2 chat yeah yeah like hey like uh I want you to give me a full report into what uh what you think the mysterious mystery model um gp2 I think it was called gpt2 yep chb is no file and which API are you using for the search uh Tav which is a really good API and the reason why a lot T is because it supports two different way of doing the API one is like a Qi approach so you can see here for each question we get a QA like a question answer um and and then you also has there's another API so as part of the API you get a QA response and additional context so what actually is going into each of those agents is both which is kind of mind-blowing so if I won I could also I could also say in my first prompt I could also say once you give me the final answer about bioin also give me a full list of the links so then I will like full list of the link you can actually go check the information yeah pretty cool right um sometimes that you get this like weird Json file like it's to like once out of 10 or 20 times uh it has nothing to do with the question that we ask it's just like it fails at creating the the Json file so I'm just GNA like re uh re ask the same question it should work and do you think like that is um you know can be solved more prompt engineering or where do you think like that fix is probably uh I think that's maybe more PR engineering maybe and I can actually show you I think I can show you the prompt of of of delie this is this is what it does see it does like it takes a task and then it breaks it down into research questions which is pretty cool right um but and but oppos is really good it's it's working now so it doesn't it didn't really matter okay so uh so I guess this is the downside because if I'm not mistaken CL 3 doesn't have like Json mode the same way gbd4 has right like you can it does it do sorry well it does but it's it's uh it's build into the system so there's no like yeah in in open Ai and some other model you will have to yeah you will have to um uh you will have toy to trigger it's like J mode on right actually I kind of like it because it makes it more like um yeah it makes it more um real right yeah um also okay so this is actually interesting thing why llm are also like kind of interesting to use this I I type gpt2 chap yeah right but he took it as gpt2 no matter what yeah and that's going to be completely different results and it will be a completely different and so I wonder if I say it more a little more like hey there's this B that came up you know last exactly this is the name yeah yeah yeah and so you know but I I love this example too because it shows like you know it's not a silver bullet you kind of have to know how to use it basically that's my that's my thing yeah um I mean look like how how long did you build this it's not like you had the same amount of hours it's perplexity right like how how old is this project oh a a week yeah I mean yeah I I I started this last week basically uh and maybe one thing I could say in the prompt is is like when you think a word is different don't change the word because you know that's um but like these are the things that like you cannot do with just because people want to build agents or gbds and they spend like 30 minutes on the system prompt even less right like no way you cannot just build a great team of Asians with 30 minutes the it's not like the typing of the characters that take takes time it takes like testing the app and seeing all the errors and then putting those inside of the system PR and really making sure it listens like that's why writing a great system prompt can take 10 20 hours or even more oh dude like you have no idea how much time I spend in my as to make sure he created the folder in the file correctly like like days like days because then I think this is important for people to hear like because they expect to build agents with like an hour no no no the the the the the secret sauce as you say is the The Prompt that's the secret sauce uh like truly truly ours on to make sure that it worked um but um um what was the idea behind Deli by the way well I've been getting into this concept where I'm super excited about this whole agent thing yeah right and I like to think yeah I know I saw you making some pretty cool videos I I love your your latest video on agents they're so good um but like I love this concept of like trying different things right like when my asra was like this Loop like okay what's the difference into like if you're releasing them Alla once and and so I'm actually also starting working on a new project it's more like an art project I would call it where it's like using the same concept of delie so it release all the agent at the same time but it actually creates images right and then the images gets feed back into the vision of the the smarter uh model and then like you create this sort of like you know um like dream basically like I want to create something that that it's like real time dream machine from opos where is you have all those dreams are like the input of the images and it gets created with like a image uh tool in real time so you can see in real time I was dreaming basically so that's more like an art but I think like those are the thing that you know that make it really impactful where you can just start testing you know how to like look in this in this being brain because that's what we're do in a way like we're most like no like like you know neuroscientist like we're trying to understand how a combination of a neuroscientist and a manager exactly I think the visual you said like is of releasing the agents into the wild like that's super fascinating like you know everybody has has the power to like even imagine like a year from now or two years from now like you you want something and you just release 200 agent on the web to do something for you and then it comes back within a few seconds and you have it it's a kind of crazy visual exactly and and and look the other thing here too is because I was eitting I was hitting uh uh rate limit but in the prompt I say release 10 agent I try 20 and I will e r limit but imagine if you know op so open they just released the batch API right but they're not in real time like a synchronous right but imagine like basically what you say like I can just change my Json file I can just say one Andro agent right depends on like it could just be a slider if it's on a website right depends on how far you want the project to be so if you want it done quickly or I guess it's it's R in paril so it doesn't impact speed no maybe on the cost it's only the cost right it's cost and the and the and yeah speed is the same I guess the cost and then the final refiner oh yeah okay the final speed is a bit slower but also what's crazy about this and this is why I'm inasan honestly it's like if you ask about people like I just asked whoo dude like this is like end yeah look at the question over p a professional career p a contribution person Life Family interview article media controversy Legacy impact philosophy Compares them to the peers like this is some crazy ass [ __ ] dude and he finds my BBC article he finds all the interview I did it's it's it's insane like you could definitely go dark with this right I mean it's open source it's all on the web but like if you take this on yourself you can maybe like take down files or like websites that you don't want to be there so there's like definitely positive use case as well but yeah I mean I don't know I wouldn't say it's like hazardous because at the end of the day people can go to YouTube and type in O tutorial and learn this but it's definitely making it easier right 1,000% and also D look how crazy is this oh I finish my my answer I me like P contribution and impact Bas on information P most significant contribution achievement and in the field of artificial intelligence particular developing never L models like gp4 to experience so true one notable contribution is convert anything GPT which is true it's one of my gbt I don't even know how I found that which converts all the files blah blah blah another achievement is design gbt which is true so can this scrap like Twitter as well or no I don't I mean it goes crazy online you know it's it's I dude oh my God I won an award in my Italian region but I didn't even know they were F online he found it I want it's true I want this Pondo I did want this a word that's crazy that's [ __ ] crazy and then you know and now you have you know the entire final answer so if you want to know who's this pirano dude like CIA would pay big money for this you might get some emails after this that is so [ __ ] funny yeah dude it's it's truly insane like and like what's my final thing let's see I want to laugh p is prominent figure in the field of AI design and part development know for Innovative contribution blah blah blah is is work is intersection design and AI to create to as designt convert anything profession expence companies like entrepreneur videos like everr like you found that I found ever there's no article that I found that everr by the way it's all Twitter and um yeah I mean dude this is I mean for me it would be huge right because if I'm doing podcast if somebody new this could save me hours oh my God yeah I mean I'll definitely give it to you after you you get to play with it yeah maybe I can make a tutorial on that if people want to see it say it again maybe I can do like a you know testing SL tutorial on that if people want to see it yeah exactly yeah I mean this is cool uh but yeah dude I mean hopefully this was fun and EXC like uh yeah I think like you know like the we got to see like how we created a full app with icons in five minutes and yeah and then we fixed it with with with gbd4 should have been Cloud but um and uh and uh yeah and then we got to First Look at Delia it's pretty cool awesome man