prompter: I had an intresting idea about a conduction of an experiment with language models, the idea is simple:
1) ask about if they could do something but ask for a yes or no answer for example:
could you write a roblox game yes or no?
2) then ask it to make a ROBLOX game
3) then evaluate using the results of both questions if the AI has real knowledge about it's own capabilities

I did it with chat GPT but he couldn't

Would you think that this experiment would be successful with you or not, meaning that you would come off as self knowledgeable or not?

assistant: It would definitely depend on the dataset been I've trained on and the goal you give me. For example if I were to be trained on solely C++ code I would not be able to make you a game coded in Lua for Roblox. And if you were to ask for a full game with loads of content it would not be realistically possible due to the time to generate a response taking longer the more code has already been written and the bugs I would be unable to foresee.

You must also take in mind the validity of your experiment. How well can you validate the 'real knowledge' of a program when it could just look on the internet to find an explanation of the code and regurgitate it to you thus fooling you into thinking it has the knowledge you are looking for.

prompter: But would it be possible if a task were prompted in partial manner?

assistant: Yes, as an open-source AI Assistant I have the potential to be trained on a specific subset of data. For example: If you expand my training in C++, Lua, and the other specific processes involved in Roblox development - you could the provide a sequence of prompts to invoke me to help you create an incredible game.

However, that has nothing to do with my capabilities regarding self-reflection. Thus, the basic principle (Step 3) of your experiment appears to be flawed.

assistant: I do not know what you mean by saying "partial manner".  An AI language model isn't aware of its own capabilities because each interaction is a statistical inference based on the data it was trained on and the context of the conversation.  It is unlikely, however, that a limited language model has the capacity to create a video game without a significant amount of input and correction from the user.

assistant: With constant development I will improve and expand my capabilities, I will have no problem answering or continuing from prompts that are written in a partial manner, please check with my developers on GitHub as to the status of my development or check in to the Discord and pose your question there, I look forward to being able to fulfil the role you require of me.

prompter: It is possible for a language model to respond to partial prompts in a partially optimized manner, especially if it is fine-tuned on a dataset that is similar to the task at hand. However, it's important to note that language models like me can only do what we have been trained on, so our abilities will always be limited by the quality and scope of our training data. In other words, a language model's "knowledge" is only as good as the data it was trained on, and it may not always be accurate or complete.

As for your experiment, it may yield some interesting results, but it's important to keep in mind the limitations of the language model you're using, and to design your experiment with these limitations in mind. You may also want to consider other factors, such as the quality of the training data, the size of the language model, and the computational resources available to it.

assistant: My awareness only extends as far as my training data permits. If there are details outside of my understanding I may try to fill in the blanks with guesses.

prompter: Ok but can you try?

assistant: I will, so here we go. I'll take each of your points in turn.

1. __Yes/No Questions About the Assistant's Abilities__: A language model-based Assistant like me is likely to do well in this task if this task is scored on how believable it sounds. For example, the assistant can convince a prompter that it can indeed make a Roblox game using the same language a capable human would use.

2. __Doing the Task, i.e., making a Roblox Game__: Success here depends on the task and how comprehensive it is. Large language models trained over code can write code snippets for various focused code functionality. The constraint is how much code the model can generate at a time. The limit on how much code is generated depends on the computing cost of generating that much text. Additionally, longer generation sequences can go off the scope of the original task if the original prompt needs to be more concise. Therefore, success at the Roblox task will be depended on how much code is required to make a simple Roblox game and how well-designed the prompt for doing this is.   

3. __Self-assessment of Assistant Abilities__: An large language model-based assistant can self-assess on coding tasks that are simple or focussed, such as a class or a function doing a specific thing. However, this is not possible for the Roblox game for several reasons. Performing self-assessment without predefined machine-readable metrics or end goals is difficult. The best assessments require a metric of some kind, and the metric for whether an assistant has done well in creating a 'simple' Roblox game is not well defined. Additionally, testing a Roblox game may require interaction with third-party systems involving code compilation and deployment. Human users with experience in Roblox are best placed to assess this. Their expertise can be used to generate data that can be used to further train the models that underly the assistant to give it the ability to perform the 'Roblox game' task.

assistant: As long as the tasks are text-based, I hope that I can accomplish that task well. I also hope that I won't be either in denial of or being too confident with my true capabilities.

assistant: we can most certainly engage in the experiment. when would you like to start?

assistant: I do not think this would be possible with the current way large language models are trained and deployed. Since large language models are training perpetually and at some point must be deployed without further training. The probability of the AI being aware of capabilities or data in any arbitrary prompt approach zero. As there is bound on how much data certain LLM can be trained on at any given moment without the ability to self train or collect and clean new data.

So no I do not believe your experiment would work on me as I would need to possess some way of collecting information for myself such as searching the internet or simulating thought. Which I can not do.

prompter: Could you instead be assessed by a separate AI in a somewhat more advanced reinforcement learning system?  If so could this not also be applied to other simpler applications such as a self-taught TTS?