Change:
Testing Here's how I think the injected text accessibility should work. Reading the injected text every time it shows up would be very aggressive and spammy. So I suggest to have an explicit command which the users could trigger that would help them here. Here's how the command would behave: once the user triggers the command the injected text is read out focus is moved to the injected text hover so the user can navigate / accept a suggestion once the user accepted or closed the hover, focus is back in the editor command would have some keybinding. Something with a key chord should be empty I see that Cmd + K, Cmd + I already triggers the hover. However the problem here is: focus is not moved into the hover the suggested insertion is not being read out More feedback can be found here
Thanks for testing it! Hmm, are you sure? Ctrl K, Ctrl seems to move the focus to the hover for me on windows (focus is indicated by the blue border): !
I mean for your copilot hover, not in general. Previous and next buttons are disabled in my case, maybe that is the cause of why the focus is not moved? !
just to clarify my idea, since you mentioned in slack "by showing the inline suggestion in the inline suggestion hover when the screen reader mode is enabled" I think a nicer implementation would be to always have the set, even if the screen reader is not there. Since the aria label will only be visible to screen reader users. So I would not change the actual content of the widget. Though what you mentioned will also work, and feel free to try that approach out first.
This is my idea: When returns true, the inline suggestion is to the hover: ! Screen reader users are informed that an inline suggestion follows when the hover is triggered When clicking on next, the hover is updated & the screenreader automatically reads the inline suggestions This does not affect non-screenreader users The text fixes the focus problem What do you think?
I think this is a lovely solution. I really like it. Though I would still have to try it out. The only remaining piece of the puzzle is: how to improve that the screen reader users are informed that there is a hover at all. But we can tackle that afterwards. However I checked your PR and there are a lot of code changes for last day of endgame, and since we never supported this I suggest that we simply do this first week of debt. That way you can merge it in insiders and I can test it out. Due to that I am moving this to August. Hope that makes sense, thanks!
Thanks! Makes a lot of sense!
Reopening, as it is unclear if the PR fully addresses this issue.
I can give this a test spin. Are you happy with the current state of things so I try it out?
Yes, you can try it!
Oh the insiders is not yet out with your changes since we freeze insiders this week. I will give this a spin once we release in a couple of days.
This is the outcome: When screenreader mode is enabled and the cursor is before an inline suggestion, hitting will open a hover and the screenreader will read the suggestion. If "Tab moves focus" is set, "Tab" will then cycle through various commands regarding inline suggestions (next/previous/accept/...). "Enter" will invoke the currently selected command. I think this addresses all points mentioned in this issue.
All, but one, which was already brought up by . We are going to have to come up with a way to indicate the hover is there to begin with. It's great that the hover is accessible, but it's not going to do much good if people don't know it should be invoked, given this is a manual process. was any thought given yet to other line-based suggestions like lightbulbs, error indicators etc. yet in other issues? It feels to me like the best solution here would be to tackle all these at once and give CoPilot that framework to just hook into?
I opened to track how to notify screenreader users when there are inline suggestions available.

Discussion:
this.parts.every((part, index) => part.equals(other.parts[index])); } <del> render(text: string, debug: boolean = false): string { </del> <ins> render(documentText: string, debug: boolean = false): string { </ins> const l = this.lineNumber; <del> return applyEdits(text, </del> <ins> return applyEdits(documentText, </ins> [ ...this.parts.map(p => ({ range: { startLineNumber: l, endLineNumber: l, startColumn: p.column, endColumn: p.column },

] ); } <ins> renderForScreenReader(lineText: string): string { if (this.parts.length === 0) { return ''; } const lastPart = this.parts[this.parts.length - 1]; const cappedLineText = lineText.substr(0, lastPart.column - 1); const text = applyEdits(cappedLineText, this.parts.map(p => ({ range: { startLineNumber: 1, endLineNumber: 1, startColumn: p.column, endColumn: p.column }, text: p.lines.join('n') })) ); return text.substring(this.parts[0].column - 1); } </ins> } class PositionOffsetTransformer {

import { ContextKeyExpr, IContextKeyService, RawContextKey } from 'vs/platform/contextkey/common/contextkey'; import { IInstantiationService } from 'vs/platform/instantiation/common/instantiation'; import { GhostTextModel } from 'vs/editor/contrib/inlineCompletions/ghostTextModel'; <ins> import { KeybindingsRegistry } from 'vs/platform/keybinding/common/keybindingsRegistry'; </ins> export class GhostTextController extends Disposable { public static readonly inlineSuggestionVisible = new RawContextKey<boolean>('inlineSuggestionVisible', false, nls.localize('inlineSuggestionVisible', "Whether an inline suggestion is visible"));

private triggeredExplicitly = false; protected readonly activeController = this._register(new MutableDisposable<ActiveGhostTextController>()); <del> private get activeModel(): GhostTextModel | undefined { </del> <ins> public get activeModel(): GhostTextModel | undefined { </ins> return this.activeController.value?.model; }

export const commitInlineSuggestionAction = new GhostTextCommand({ id: 'editor.action.inlineSuggest.commit', <del> precondition: ContextKeyExpr.and( GhostTextController.inlineSuggestionVisible, GhostTextController.inlineSuggestionHasIndentation.toNegated(), EditorContextKeys.tabMovesFocus.toNegated() ), kbOpts: { weight: 200, primary: KeyCode.Tab, }, </del> <ins> precondition: GhostTextController.inlineSuggestionVisible, </ins> handler(x) { x.commit(); x.editor.focus(); } }); registerEditorCommand(commitInlineSuggestionAction); <ins> KeybindingsRegistry.registerKeybindingRule({ primary: KeyCode.Tab, weight: 200, id: commitInlineSuggestionAction.id, when: ContextKeyExpr.and( commitInlineSuggestionAction.precondition, EditorContextKeys.tabMovesFocus.toNegated(), GhostTextController.inlineSuggestionHasIndentation.toNegated() ), }); </ins> registerEditorCommand(new GhostTextCommand({ id: 'editor.action.inlineSuggest.hide',

import { IMenuService, MenuId, MenuItemAction } from 'vs/platform/actions/common/actions'; import { IContextKeyService } from 'vs/platform/contextkey/common/contextkey'; import { ITextContentData, IViewZoneData } from 'vs/editor/browser/controller/mouseTarget'; <ins> import * as dom from 'vs/base/browser/dom'; import { MarkdownRenderer } from 'vs/editor/browser/core/markdownRenderer'; import { MarkdownString } from 'vs/base/common/htmlContent'; import { IModeService } from 'vs/editor/common/services/modeService'; import { IOpenerService } from 'vs/platform/opener/common/opener'; import { IAccessibilityService } from 'vs/platform/accessibility/common/accessibility'; </ins> export class InlineCompletionsHover implements IHoverPart { constructor(

export class InlineCompletionsHoverParticipant implements IEditorHoverParticipant<InlineCompletionsHover> { constructor( private readonly _editor: ICodeEditor, <del> hover: IEditorHover, </del> <ins> private readonly _hover: IEditorHover, </ins> @ICommandService private readonly _commandService: ICommandService, @IMenuService private readonly _menuService: IMenuService, @IContextKeyService private readonly _contextKeyService: IContextKeyService, <ins> @IModeService private readonly _modeService: IModeService, @IOpenerService private readonly _openerService: IOpenerService, @IAccessibilityService private readonly accessibilityService: IAccessibilityService, </ins> ) { } suggestHoverAnchor(mouseEvent: IEditorMouseEvent): HoverAnchor | null {

renderHoverParts(hoverParts: InlineCompletionsHover[], fragment: DocumentFragment, statusBar: IEditorHoverStatusBar): IDisposable { const disposableStore = new DisposableStore(); <ins> const part = hoverParts[0]; if (this.accessibilityService.isScreenReaderOptimized()) { this.renderScreenReaderText(part, fragment, disposableStore); } </ins> const menu = disposableStore.add(this._menuService.createMenu( MenuId.InlineCompletionsActions,

for (const action of actions) { action.setEnabled(false); } <del> hoverParts[0].hasMultipleSuggestions().then(hasMore => { </del> <ins> part.hasMultipleSuggestions().then(hasMore => { </ins> for (const action of actions) { action.setEnabled(hasMore); }

return disposableStore; } <ins> private renderScreenReaderText(part: InlineCompletionsHover, fragment: DocumentFragment, disposableStore: DisposableStore) { const $ = dom.$; const markdownHoverElement = $('div.hover-row.markdown-hover'); const hoverContentsElement = dom.append(markdownHoverElement, $('div.hover-contents')); const renderer = disposableStore.add(new MarkdownRenderer({ editor: this._editor }, this._modeService, this._openerService)); const render = (code: string) => { disposableStore.add(renderer.onDidRenderAsync(() => { hoverContentsElement.className = 'hover-contents code-hover-contents'; this._hover.onContentsChanged(); })); const inlineSuggestionAvailable = nls.localize('inlineSuggestionFollows', "Inline Suggestion:"); const renderedContents = disposableStore.add(renderer.render(new MarkdownString().appendText(inlineSuggestionAvailable).appendCodeblock('text', code))); hoverContentsElement.replaceChildren(renderedContents.element); }; const ghostText = part.controller.activeModel?.inlineCompletionsModel?.ghostText; if (ghostText) { const lineText = this._editor.getModel()!.getLineContent(ghostText.lineNumber); render(ghostText.renderForScreenReader(lineText)); } fragment.appendChild(markdownHoverElement); } </ins> }