Repository: Azure-Samples/cognitive-services-speech-sdk

Issue #1277: Java SDK - startContinuousRecognitionAsync never stops
Created: 2021-10-07T03:25:18Z
I am using the Java SDK and have been attempting to find out why the speechEndDetected never occurs.  I've used an PushAudioInputStream to send a WAV file.  It transcribes the whole thing but never sends the stop event, which means I can't get the response back reliably.
If I don't wait for some kind of stop event, then it returns a partial result instead of the whole thing.
To Reproduce
Steps to reproduce the behavior:

Create Java Spring API
Create Controller endpoint

@PostMapping(path = "/transcribe", consumes = { MediaType.MULTIPART_FORM_DATA_VALUE })
	public String uploadFile(@RequestPart(value = "file", required = true) MultipartFile files)
			throws InterruptedException, ExecutionException, IOException {

		AudioStreamFormat format = AudioStreamFormat.getWaveFormatPCM(40000L, (short) 16, (short) 2);
		PushAudioInputStream audioStream = AudioInputStream.createPushStream(format);
		InputStream fileStream = files.getInputStream();
		audioStream.write(files.getBytes());

		AudioConfig audioConfig = AudioConfig.fromStreamInput(audioStream);
		SpeechRecognizer recognizer = new SpeechRecognizer(speechConfig, audioConfig);

		try {
			Semaphore stop = new Semaphore(0);
			recognizer.startContinuousRecognitionAsync().get();

			StringBuilder text = new StringBuilder();

			recognizer.recognized.addEventListener((o, e) -> {
				if (e.getResult().getReason() == ResultReason.RecognizedSpeech) {
					System.out.println("RECOGNIZED: Text=" + e.getResult().getText());
					text.append(e.getResult().getText());
				} else if (e.getResult().getReason() == ResultReason.NoMatch) {
					System.out.println("RECOGNIZED: Text=" + e.getResult().getText());
					text.append("NoMatch");
				}
			});
			recognizer.recognizing.addEventListener((o, e) -> {
				System.out.println("RECOGNIZING: Text=" + e.getResult().getText());
			});
			recognizer.speechStartDetected.addEventListener((o, e) -> {
				System.out.println("SPEECH START DETECTED");
			});
			recognizer.speechEndDetected.addEventListener((o, e) -> {
				System.out.println("SPEECH END DETECTED");
				stop.release();
			});
			recognizer.canceled.addEventListener((o, e) -> {
				text.append("Failed:" + e.getReason() + " [" + e.getErrorCode() + "] - " + e.getErrorDetails());
				stop.release();
			});
			recognizer.sessionStarted.addEventListener((o, e) -> {
				System.out.println("SESSION STARTED");
			});
			recognizer.sessionStopped.addEventListener((o, e) -> {
				System.out.println("SESSION STOPPED");
				stop.release();
			});

			stop.acquire();
			recognizer.stopContinuousRecognitionAsync().get();

			audioStream.close();
			fileStream.close();

			return text.toString();
		} finally {
			audioConfig.close();
			recognizer.close();
		}
	}
Expected behavior
When the SpeechRecognizer no longer has anything more to transcribe it should send a stop event.
Version of the Cognitive Services Speech SDK
1.18.0
Platform, Operating System, and Programming Language

OS: Windows, within Docker container mcr.microsoft.com/vscode/devcontainers/java
Hardware - x64
Programming language: Java
Browser: Postman

There is no error message, it just never ends.

Duplicate issue #1411: Event SpeechEndDetected doesn't raise.
Created: 2022-02-13T09:44:09Z
I have a simple text to SRT function(Text to speech, then speech to text), and I use TaskCompletionSource to notify the end of the task, but SpeechEndDetected doesn't seem to be triggered.
P.S: I used to save the synthesized speech as a wav file first and it worked fine. Now, I'm operating directly on the memory stream and I've found this problem.
var subscription = new AzureSubscription("<key>", "<region>");
var speechService = new SpeechService(subscription, "en-US", "en-US-SaraNeural");

bytes= await speechService.CreatBytesAsync(Ssml);

var srtService = new SrtService(speechService.SpeechConfig);
using var audioInputStream = AudioInputStream.CreatePushStream();
audioInputStream.Write(bytes);

using var audioConfig = AudioConfig.FromStreamInput(audioInputStream);
var subtitles = await srtService.BuildSrtAsync(audioConfig);
public async Task<byte[]> CreatBytesAsync(Ssml ssml)
{
    using var synthesizer = new SpeechSynthesizer(SpeechConfig, null);
    var result = await synthesizer.SpeakSsmlAsync(ssml.Raw);
    byte[] stream = result.AudioData;

    return stream;
}
public async Task<List<Subtitle>> BuildSrtAsync(AudioConfig audioConfig)
{
    var recognizer = new SpeechRecognizer(SpeechConfig, audioConfig);
    var subtitles = new List<Subtitle>();

    var begin = TimeSpan.Zero;
    var end = TimeSpan.Zero;

    recognizer.SessionStarted += (sender, e) => {
        Debug.WriteLine(">>> Srt SessionStarted.");
    };

    recognizer.SpeechStartDetected += (sender, e) => {
        Debug.WriteLine(">>> Srt SpeechStartDetected.");
    };

    recognizer.Recognizing += (sender, e) => {
        Debug.WriteLine(">>> Srt Recognizing.");
    };

    recognizer.Recognized += (sender, e) => {
        Debug.WriteLine(">>> Srt Recognized.");
        begin = TimeSpan.FromMilliseconds(e.Result.OffsetInTicks / 10000);
        end = begin + e.Result.Duration;
        var subtitle = new Subtitle(e.Result.Text, begin, end);
        subtitles.Add(subtitle);
    };

    recognizer.SessionStopped += async (sender, e) => {
        Debug.WriteLine(">>> Srt SessionStopped.");
        await recognizer.StopContinuousRecognitionAsync();
    };
    recognizer.Canceled += async (sender, e) => {
        Debug.WriteLine(">>> Srt Canceled.");
        await recognizer.StopContinuousRecognitionAsync();
    };
    recognizer.SpeechEndDetected += (sender, e) => {
        Debug.WriteLine(">>> Srt SpeechEndDetected.");
        _tcs.TrySetResult(true);
    };

    await recognizer.StartContinuousRecognitionAsync();

    await _tcs.Task;

    return subtitles;
}
Output:
>>> Srt SessionStarted.
>>> Srt SpeechStartDetected.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognized.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognized.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognized.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognized.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognized.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognizing.
>>> Srt Recognized.

Closing comment (2022-02-13T10:08:00Z): Following the guidelines of this Duplicate of #1277, I added a line of code:
audioInputStream.Write(bytes);
audioInputStream.Close(); // add close()
This did solve my problem, but I don't know if this is the right "pattern", so if anyone knows, please let me know so I can do it with confidence.