Change:
We have a great visual indicator now when a speech to text session is in progress as demoed here Placeholder text appears with and the microphone icon is illuminated when the session is in progress. We should indicate this to screen reader users as well with an audio cue / alert on progress start / stop.
is this something you could work on soon?
also cc and for feedback
That video link isn't working for me -- any way I could check it out another way to get context? Would this example I found make sense for me to reference? I'd want to clarify a few things but can definitely start working on this and prioritize this in the queue if needed! Let's connect Monday to discuss.
Yes, the link you have provided is correct
I think once we determine questions surrounding this and WIP Alert Progress Status Audio Cue , we can set forth something that most efficiently supports all of our progress indicator use cases (CoPilot chat, Text to Speech, Cell Execution, Settings Search, etc). A global progress system (with a unified/associated sound) might have multiple subset playback systems (e.g. looping for CoPilot Chat, one shots, incremental tones, etc), based on needs. Plus additional audio cues (success/fail), and Aria alerts, depending on need for context switching and other variables. • Do we need both 1) "Listening" audio cue and 2) a "Progress Indicator" sound? • How likely is it that the experience would be similar to a traditional chat, where you might hear a burst of repetitions, or a few in succession every .5 to 4 seconds? • What are the wait times we can expect here? • How likely is the need for context/window-switching?
Thanks for the great questions Yes, because playing a sound while a speech to text recording is in progress would be confusing IMO. This will vary based on the use case, but my feeling is successive repetition will not happen often. Anywhere from < 1s to upwards of 30s Could you pls clarify what you mean here?
Thanks! Re "context/window-switching" clarification: what's the likelihood of needing to switch windows to work on something else while waiting for progress to finish, in parallel? [For example, in CoPilot Chat, it was determined that context switching was likely, and we implemented an Aria alert (not sure if it ended up being default or opt-in, post screen reader detection)]
thanks for clarifying. I think this will depend on the particular context, but we should account for the possibility of context switching, especially as the progress time increases.
one thing that came up with my 1on1 with : should this audio cue always play, even when accessibility mode is not on? this could give a nice indicator when you can actually start talking, i.e. when the mic is ready to accept input. Not sure if we have a capability today to play an audio queue always.
Yes, we'd simply set the default value to and this could work.
would be great to do this in February, see above for discussion.
For the cue that indicates the completion of recording, I worry about the media policy related to auto-play causing it to not play cc perhaps we could investigate fixing that in February? reverted the fix for that issue here
I can definitely roadmap this for February! End of Feb would be ideal. This feels like 3, not 2 sounds atm, so, same as the og Copilot chat flow but instead of a "Send" sound, it's a "Listening" sound, yes? What do you think of treating it like that (see below)? (Btw, I'm working on some subtle improvements for existing Copilot Chat Send/Progress/Receive sounds. This would be a good place to implement) Sound 1 Listening (Default On): a new sound, which plays w/visual feedback “I’m listening”/mic animation Sound 2 Progress Start (Opt-In): plays w/ text “Fetching Response” Sound 3 Response (Opt-In): plays at Copilot response Where do you all see ARIA alerts fitting into this — same as existing Copilot chat usage?
We will not use alerts as they'll mess with the microphone. I think those sound good
et all, a few more qs:) Can VS Code Speech only be used in chat interfaces provided by Copilot Chat extension? Appears that way from the documentation... It's not available for Inline Suggestions, correct? Come to think of it, if it's essentially speech to text exclusively for Copilot Chat's 3 views (Inline Chat, Quick Chat, Chat View), is there any reason to not fully treat this as an improved/extended o.g. Copilot Chat Audio experience, and provide the fourth (opt-in) "Send" sound (which already exists)? It'd be overkill to use all four for most, but wonder if the flexibility to mix/match outweighs that, given all the variables of screen readers, and consistency w/the o.g. Copilot Chat flow
Currently the extension is used in all copilot chat contexts but we plan on bringing it to terminal and editor maybe later.
do you agree we should have a start and a stop recording sound instead of one that plays throughout the recording?
yes!
cc
Oh yah, we're in agreement here -- I wasn't suggesting a Progress sound to take place during any recording. With regard to "progress start / stop" being mentioned, I was referring to the Copilot Progress sound which could happen after recording has stopped. So, yes, two Listening cues (Start & Stop Listening) instead of one could make sense to me. I'm thinking that if there's any default set to "on" for "no screen reader detected", it should only be for Start, not Stop. This is because that user is likely to know when the recording is over. For "screen reader detected," the Progress sound could kick in, as well as other Copilot sounds -- what do you think? (Btw, I reached out to the MS Audio Designer community and that was their recommendation as well. I can send some further research offline as well.) Hoping to still employ a less is more approach here, but lmk what you think, and whether I'm understanding this right?
Updated flow for a Copilot Chat context — what do you think? et all ? :) -Sound 1 Listening Start (Default On): a new sound, which plays w/visual feedback “I’m listening”/mic animation -Sound 2 Listening Stop (Default Off?): a new sound, which plays when mic stops listening -Sound 3 Copilot Send sound (Opt-In): triggered by sending the query -Sound 4 Copilot Progress Start (Opt-In): plays w/ text “Fetching Response” same as Copilot progress -Sound 5 Copilot Response (Opt-In): plays at Copilot response -No additional Aria Alerts so as to not confuse w/screen reader
Sounds good, not sure yet what we will do about default for listening stop. I believe sounds 3-5 already exist.
Note to self also add this for Voice Dictation in editor/terminal
Just messaged you on Teams w/a question :)

Discussion:
public static readonly clear = Sound.register({ fileName: 'clear.mp3' }); public static readonly save = Sound.register({ fileName: 'save.mp3' }); public static readonly format = Sound.register({ fileName: 'format.mp3' }); <ins> public static readonly voiceRecordingStarted = Sound.register({ fileName: 'voiceRecordingStarted.mp3' }); </ins> private constructor(public readonly fileName: string) { } }

settingsKey: 'accessibility.signals.format' }); <ins> public static readonly voiceRecordingStarted = AccessibilitySignal.register({ name: localize('accessibilitySignals.voiceRecordingStarted', 'Voice Recording Started'), sound: Sound.voiceRecordingStarted, legacySoundSettingsKey: 'audioCues.voiceRecordingStarted', settingsKey: 'accessibility.signals.voiceRecordingStarted' }); </ins> private constructor( public readonly sound: SoundSource, public readonly name: string,

}, } }, <ins> 'accessibility.signals.voiceRecordingStarted': { ...defaultNoAnnouncement, 'description': localize('accessibility.signals.voiceRecordingStarted', "Indicates when the voice recording has started."), 'properties': { 'sound': { 'description': localize('accessibility.signals.voiceRecordingStarted.sound', "Plays a sound when the voice recording has started."), ...soundFeatureBase, 'default': 'on' }, } }, </ins> 'accessibility.signals.clear': { ...signalFeatureBase, 'description': localize('accessibility.signals.clear', "Plays a signal when a feature is cleared (for example, the terminal, Debug Console, or Output channel)."),

import { ISpeechService, ISpeechProvider, HasSpeechProvider, ISpeechToTextSession, SpeechToTextInProgress, IKeywordRecognitionSession, KeywordRecognitionStatus, SpeechToTextStatus, speechLanguageConfigToLanguage, SPEECH_LANGUAGE_CONFIG } from 'vs/workbench/contrib/speech/common/speechService'; import { ITelemetryService } from 'vs/platform/telemetry/common/telemetry'; import { IConfigurationService } from 'vs/platform/configuration/common/configuration'; <ins> import { AccessibilitySignal, IAccessibilitySignalService } from 'vs/platform/accessibilitySignal/browser/accessibilitySignalService'; </ins> export class SpeechService extends Disposable implements ISpeechService {

@IContextKeyService private readonly contextKeyService: IContextKeyService, @IHostService private readonly hostService: IHostService, @ITelemetryService private readonly telemetryService: ITelemetryService, <del> @IConfigurationService private readonly configurationService: IConfigurationService </del> <ins> @IConfigurationService private readonly configurationService: IConfigurationService, @IAccessibilitySignalService private readonly accessibilitySignalService: IAccessibilitySignalService </ins> ) { super(); }

if (session === this._activeSpeechToTextSession) { this.speechToTextInProgress.set(true); this._onDidStartSpeechToTextSession.fire(); <ins> this.accessibilitySignalService.playSignal(AccessibilitySignal.voiceRecordingStarted); </ins> } break; case SpeechToTextStatus.Recognizing: