Code change:
This issue is about allowing extensions to use language model that are contributed by other extensions. The API proposal is and its scope is to allow to use the chat feature of language models. Remaining finalisation todos [x] consolidate different LanguageModelChatXYZMessage-type into one type and a role-enum [x] move System message/role into its own proposal [x] have a select-function that's the only way of getting to a chat model [x] [ ] consider to have more sturcture on , e.g have a fake reply message with role like others (ollama, open ai, anthropic) I renamed to and I will continue to explore a stream of structured objects. That would be additive to and plugged in as its foundation. add something like add a StopReason-enum that's also used in LanguageModelChat
Docs:
With there will be no more language model access object. Instead all requests are made directly via
Hi Is there any plans to add a , so I can implement my own calls to other models other than the Copilot provided models? Thanks!
Yeah, that exists. It is the which is still very rough and currently not yet scheduled for finalisation. Tho, feel free to give it a spin and report what's missing/wrong. For instance, locally I have a dummy extension that uses ollama to drive this API
I assume you are referring to the API? So if I use , I can then use the id I give it with ?
No, I mean the proposal that I have mentioned
FYI that I made the following changes to the proposal there is no more and generally no more referring of language model by string-identifier. there is that takes a selector and returns matching chat models or nothing () These changes allow for a better implementation of the providing-end (-proposal) and most importantly make it more obvious that models can (and will) change. The following pattern had become popular'ish , ....)`. This is brittle because extensions break as soon as the model id isn't known anymore. Instead models need to be selected, making it hopefully more clear that a desired model doesn't exist. The new selector-mechanism allows for new flexibility as well. A couple of samples To restate, the -function can return which means no model was found. That's because the selector is to strict (using ) or because no models are known (missing extensions). The shape of language model selectors is this Note that property order represents how specific it is properties are matched based on equality, when present it must match we will likely add some comparison selector for input and/or output tokens
Isn't this still possible? We can call any LLM API in our handler and use the stream.markdown() to display the response, right?
'y.provider.provideLanguageModelResponse is not a function'
I'm noticing a difference in behavior of the API when running in Codespaces from the browser. When running against a Codespace with desktop Insiders, everything works as expected. When running against the same Codespace with browser Insiders, always returns , implying there are no chat models available.
Make sure that you are on latest pre-release of the copilot-chat extension That's certainly possible when copilot-chat isn't installed in the codespace.
It is installed, and regular Copilot Chat works normally, but our extension can't get a language model.
And it is latest prerelease?
Yep! Version: 1.90.0-insider Commit: Date: 2024-05-13T12:40:35.552Z Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0 ! ! Here's regular chat working: ! And our extension emitting to its output log, via this line of code: ! The same codespace but using desktop frontend instead of browser (working as expected): !
Really confusing... In my sample remote-setup (which isn't codespaces) this is working just fine and I see so smoking guns for why this wouldn't work in codespaces. For now, I have some logging that should help us get to the bottom of this
I see mention of moving the System prompt to a separate Proposal, can that be linked to? I am not finding it and we use it in our extension. Also is this section of the guide still accurate? It's not showing those commands as imports anymore and from my reading of the types it looks like it should be closer to:
This is place of all API proposals: Likely outdated
Where should I look for the additional logs?
Enable trace logging, show the window-log and check for messages starting with . There should be some for registering and some for selection <img width="963" alt="Screenshot 2024-05-15 at 08 15 30" src=""
I did find something after running into a similar issues. Can you confirm that a later/second invocation yields models and/or that the -event fires and that after that chat models are selected?
I found some log lines for selection but nothing for registration: We end up doing language model selection several times and each time it returns . I even tried doing the following and still there were no log lines about registration: all 3 extensions (Copilot Chat, Copilot, and our extension) browser window trace window logging extensions a query with regular Copilot (success) a query with our extension (failure) However, when using VSCode Desktop as the frontend, I do see log lines about language model registration:
Thanks for the update. That would really mean the language model API doesn't register. Can you check the chat log (Output GH Copilot Chat) for messages starting with
I don't see any messages like that from either browser or desktop frontend on the Codespace. However, on the browser session there was this message which did not appear on the desktop session--might be spurious but I'm not sure:
We have finalized the Language Model and Chat Participant API Thus, you can now publish your extensions to the VS Marketplace. This API is finalized in VS Code Insiders, and will be finalized in VS Code Stable in July. Thus we suggest that you use an engines: ^1.90.0 property in your , and VS Code Stable will gracefully handle your extensions and ignore your usage of Language Model and Chat API until the API gets finalized in Stable in July. Full docs We have decided to only finalise the message. And message might be finalised in one of the future iterations. Main reason for this is general API consideration - as we are still learning how the prompt API would fit with some of the non-GPT models. Also we believe that the prompt API requires more responsible AI testing from our team. If your extension would like to use the prompt API - please let us know if you hit issues with using the prompt as a replacement. We are excited to see what you build
I am still wondering if there will be a API so I can use other models inside of the chat API. Is this something coming in the future? For now, it can be shimmed by wrapping and handling other entries (perhaps I want to point to my local Ollama instance).
Still a proposal and not yet planned for finalisation

Related discussion:
text: await barrier.wait(); if (error) { <del> throw new Error(`Language model '${languageModelId}' errored, check cause for more details`, { cause: error }); </del> <ins> throw new LanguageModelError( `Language model '${languageModelId}' errored, check cause for more details`, 'Unknown', error ); </ins> } return res.apiObject;