Change:
There are currently two issues when you try posting a ArrayBuffer to a webview or from a webview back to VS Code: is not revived. Instead the receiver ends up getting a JavaScript object literal with the contents of the array buffer messages for typed arrays are currently super inefficient . For example, a Uint8Array is serialized as In , add a second parameter that lets extensions specify the types to 'transfer': This is inspired by the , however to start with, we'd only allow transferring of array buffers. When is provided, we'd walk the object to find any object that appear in the message and make sure these are more efficiently transferred. We'd also revive them properly on the other side (so if an goes in, then an object comes out too) We'd potentially break existing extensions that rely on the current serialized format of ArrayBuffers/typed arrays. Having a parameter also lets us really transfer the in cases where this is possible No, we can't. Consider an extension running on a remote with a local webview for example. There is no way to actually transfer the between these contexts. Instead we have to serialize it and then recreate it on the other side. However, this new api would still let us improve the serialization and reduce the amount of data transferred. Even if we can't really transfer ownership, semantically, should behave like in that extensions should not use transferred objects after they have transferred. I'm not sure if we can enforce this though Potentially, but not all of them. Because we can't support transfer all the time (see above), we can only support types that can be serialized. This means no or other rich objects.
Copying a few people who may be interested in this: for GitLens (or CodeStream contacts) for LunaPaint for notebooks (I think we'd have to add this feature to the notebook api as well) for DrawIO
also cc Personally I find the argument rather ugly. I'd rather expose a class that would be able to wrap user types, like . However the argument is what native postMessage does, so I grudgingly agree that we should match it.
While I like the inspiration of the transfer parameter, we might need a different name as that's specifically transferring ownership of the buffer so it can no longer be used from where you're posting it from: So it's not transferring in our case, it's more copying quickly. Also it would be convenient for the extension if typed arrays still get transferred correctly, : We wouldn't be transferring the typed array view but recording the type and recreating it on the other side using the . This would just be a really nice convenience and close the gap with the native transfer.
Current implementation is in if you want to test it out. I like the idea of reviving typed arrays too. I'll take a look to see what that would involve. Again, I think we'd only be able to enable this behavior if an extension opts in to it. Otherwise we may break existing extensions. Also on the web, we can actually transfer ownership between the extension host worker and the main process. On desktop, that's currently not possible but I do like using the transfer semantic from web so that we can potentially use them more widely in the future
Definitely agree with about trying to use as the signal to this better serialization behavior (as it break the intent when you actually want to transfer ownership). As for backward compatibility -- could we do something unorthodox, and say if an extension explicitly targets 1.54 or greater, we opt them into the new behavior, and if not they stick with the current? Because I can't see a case where anyone would want the current behavior (other than not break assumptions right now) going forward.
Thanks for the mention! My input on this is: I generally like the idea For the drawio extension, I already use a Base64 encoding to send ArrayBuffers. Would be interesting to see how much this API further increases performance of data transfer. I find it spooky that passing a transfer array has an effect on how is serialized. Usually, I would expect to be the same as . If backwards compatibility is a real issue here since someone actually put an ArrayBuffer in msg, I guess this spooky behavior will be observed and might cause bugs. It would be interesting to report an analytics event every time someone actually passes an array buffer. I doubt this is being done, in this case backwards compatibility would not be an issue. Otherwise I find it cleaner if ArrayBuffers in msg are never serialized, only the ones in transfer. I really like s idea of having different vscode apis for different extensions, depending on their targeted vscode version. However, I doubt this can be implemented without ugly and slow hacks like using the stack trace to figure out which extension calls an API.
I am also a fan of leveraging the vscode engine version as a means to do this "soft api breakage", imo since the current behavior is pretty unusable and most extensions would have a workaround anyway (, base64, etc.). AFAIK we haven't done this before though and I'm not sure if it's easy to implement, we definitely don't want to create a stack. An alternative if we want to just do this for all array buffers is to use options: There may be special cases where extensions are transferring so much data that we don't want to walk the object though?
now serializes and restores typed arrays too (, ....). If you have a chance, please test out the branch and let me know if there are any bugs and what the perf looks like for real world use cases
would be really nice if SharedArrayBuffers would be supported as well!
I update the extension api to change its behavior based on the engine version (you need to target 1.55+) On the webview side, you should still pass a parameter to . It will not effect serialization but will improve performance
so sorry for not looking at this even though i was pinged. Thanks for this, i'll check this out. In WebView today we manually serialize Array buffers to overcome this issue. However we have both types & . Will this new serializer implementation support as well? FYI - In Notebooks, IPYWidgets can end up sending array buffers to & fro.
thanks for pinging, this is useful for us (Jupyter extension with IPYWidgets).
We should serialize and other typed arrays. However is currently not supported
Can you verify with Luna paint when you have a chance?
I've tried to verify this 3 times over the past few weeks and failed each time. The serialization/deserialization code got a little complex in my codebase but I don't believe this is fixed. Here are problems I was having: You mentioned above but it's not in the types, it's not clear exactly what I need to do here Passing either a Uint8Array or ArrayBuffer from the ext host to webview ends up coming back as . This is the format of the message I was posting: <img width="486" alt="Screen Shot 2021-04-30 at 8 04 54 AM" src="https://user-"I had similar problems getting webview -ext host to work but didn't check that this time.
Thanks for checking There seem to be two issues: It still required proposed api. This was just an oversight Only array buffers can be posted to the webview (not typed arrays). I think I had a merge conflict or left some bad code in after refactoring, because there are basically two function that do the same thing, but one is correct and the other is doing things incorrectly the old way. Will update to use the correct one
Are there any plans to support dataviews? This is something Jupyter notebooks uses for ipywidgets. Or should I file a separate issue for that?
Yes please file a separate issue
I'm going to push this to 1.57 so we can get more testing to make sure it doesn't cause any regressions
To verify: and run webview sample the extension development window, run the command the developer tools in the cat coding panel, run the The branch makes this command post a message with a UInt8Array to and from the webview the developer tools, you should now see: ! first message is from inside the webview. It shows that a has been received from the extension second message is posted from the webview to the extension host. It shows that a has been correctly posted over , try downgrading the engines version in to the same steps. Now you should see: The object in the webview is now an object instead of a The type printed in the extension host is instead of
I'll see if I can test this on the weekend if all I was missing was the enable proposed api flag Thanks for the sample!
Still doesn't seem to work with this setup: Same thing for vscode engine exthost: ! webview: !
Try with 1.57+ in the current OSS build (or first 1.57 insiders) I moved this out of 1.56 because it's not urgent and I want more testing for potential regressions
Using 1.57.0 and enableProposedApi the transport time on save (webview -exthost) drops from ~4s to ~400ms. For opening (exthost -webview), it went from ~10s to ~500ms . so this is awesome and verified, but one complications I ran into and a reason why it was so hard to verify was that returned a which doesn't get the perf benefits, so the typings of readFile are lying: <img width="830" alt="Screen Shot 2021-05-30 at 7 58 42 AM" src="https://user-"Needed to do this to fix it: The type should probably be checked and handled automatically.
is any issue tracking the finalization of this?

Discussion:
import { Emitter } from 'vs/base/common/event'; import { Disposable } from 'vs/base/common/lifecycle'; import { IExtensionDescription } from 'vs/platform/extensions/common/extensions'; <del> import { ExtHostWebview, ExtHostWebviews, toExtensionData } from 'vs/workbench/api/common/extHostWebview'; </del> <ins> import { ExtHostWebview, ExtHostWebviews, toExtensionData, shouldSerializeBuffersForPostMessage } from 'vs/workbench/api/common/extHostWebview'; </ins> import { checkProposedApiEnabled } from 'vs/workbench/services/extensions/common/extensions'; import { ViewBadge } from 'vs/workbench/api/common/extHostTypeConverters'; import type * as vscode from 'vscode';

this._viewProviders.set(viewType, { provider, extension }); this._proxy.$registerWebviewViewProvider(toExtensionData(extension), viewType, { retainContextWhenHidden: webviewOptions?.retainContextWhenHidden, <del> serializeBuffersForPostMessage: false, </del> <ins> serializeBuffersForPostMessage: shouldSerializeBuffersForPostMessage(extension), </ins> }); return new extHostTypes.Disposable(() => {