Repository: Azure-Samples/cognitive-services-speech-sdk

Issue #1054: No punctuation and capitalisation in nbest (words) results
Created: 2021-04-22T13:20:25Z
Hi, I'm currently using the batch transcription REST API (2.0) and I noticed that inside Words it doesn't have punctuation or capitalisation. Any suggestion on how to receive the NBest results (list of words) with correct capitalisation and punctuation?
Current behaviour:
                    "NBest": [
                        {
                            "Confidence": 0.812473,
                            "Lexical": "hello this is jeff",
                            "ITN": "hello this is jeff",
                            "MaskedITN": "Hello this is Jeff",
                            "Display": "Hello, this is Jeff.",
                            "Sentiment": {
                                "Negative": 0.280599,
                                "Neutral": 0.719372,
                                "Positive": 0.0,
                            },
                            "Words": [
                                {
                                    "Word": "hello",
                                    "Offset": 1000000,
                                    "Duration": 3700000,
                                    "OffsetInSeconds": 0.1,
                                    "DurationInSeconds": 3.7,
                                    "Confidence": 0.882375,
                                },
                                {
                                    "Word": "this",
                                    "Offset": 38000000,
                                    "Duration": 4000000,
                                    "OffsetInSeconds": 3.8,
                                    "DurationInSeconds": 0.4,
                                    "Confidence": 0.944702,
                                },
                                {
                                    "Word": "is",
                                    "Offset": 42000000,
                                    "Duration": 1000000,
                                    "OffsetInSeconds": 4.2,
                                    "DurationInSeconds": 0.1,
                                    "Confidence": 0.900596,
                                },
                                {
                                    "Word": "jeff",
                                    "Offset": 43000000,
                                    "Duration": 4000000,
                                    "OffsetInSeconds": 4.3,
                                    "DurationInSeconds": 0.4,
                                    "Confidence": 0.900095,
                                },
                            ],
                        }
                    ],

What I want:
                    "NBest": [
                        {
                            "Confidence": 0.812473,
                            "Lexical": "hello this is jeff",
                            "ITN": "hello this is jeff",
                            "MaskedITN": "Hello this is Jeff",
                            "Display": "Hello, this is Jeff.",
                            "Sentiment": {
                                "Negative": 0.280599,
                                "Neutral": 0.719372,
                                "Positive": 0.0,
                            },
                            "Words": [
                                {
                                    "Word": "Hello,",
                                    "Offset": 1000000,
                                    "Duration": 3700000,
                                    "OffsetInSeconds": 0.1,
                                    "DurationInSeconds": 3.7,
                                    "Confidence": 0.882375,
                                },
                                {
                                    "Word": "this",
                                    "Offset": 38000000,
                                    "Duration": 4000000,
                                    "OffsetInSeconds": 3.8,
                                    "DurationInSeconds": 0.4,
                                    "Confidence": 0.944702,
                                },
                                {
                                    "Word": "is",
                                    "Offset": 42000000,
                                    "Duration": 1000000,
                                    "OffsetInSeconds": 4.2,
                                    "DurationInSeconds": 0.1,
                                    "Confidence": 0.900596,
                                },
                                {
                                    "Word": "Jeff.",
                                    "Offset": 43000000,
                                    "Duration": 4000000,
                                    "OffsetInSeconds": 4.3,
                                    "DurationInSeconds": 0.4,
                                    "Confidence": 0.900095,
                                },
                            ],
                        }
                    ],

These are the properties that I'm using:
        properties = {
            "ProfanityFilterMode": "None",
            "PunctuationMode": "Automatic",
            "AddWordLevelTimestamps": "True",
            "AddDiarization": "True",
        }

Thank you in advance,
Marina Haack

Duplicate issue #1677: No punctuation
Created: 2022-09-30T16:47:14Z
Hello,
I don't find any way to activate the punctuation on the continuous speech service. I don't see any punctuation on words nor on Display or DisplayText fields in the results.
I don't understand if this feature is activated by default or must be activated, I try setting different SpeechConfig fields to activate it.
Here is what I tried:
setting
       speechConfig.setServiceProperty('punctuation', 'explicit', SpeechSDK.ServicePropertyChannel.UriQueryParameter);
and changing the endpoint which contains /conversation in my case (I didn't figured out why) by /dictation
speechConfig = SpeechSDK.SpeechConfig.fromEndpoint(new URL("wss://westeurope.stt.speech.microsoft.com/speech/recognition/dictation/cognitiveservices/v1"))
I am guessing this is possible because on this page it works!:
https://azure.microsoft.com/fr-fr/products/cognitive-services/speech-to-text/#features, it even has a checkbox with "enable punctuation".
Here is my final non working SpeechConfig:
    function getSpeechConfig() {
        let speechConfig: SpeechConfig;

        speechConfig = SpeechSDK.SpeechConfig.fromEndpoint(new URL("wss://westeurope.stt.speech.microsoft.com/speech/recognition/dictation/cognitiveservices/v1"))
        speechConfig.authorizationToken = tokenRegion[0]

        speechConfig.outputFormat = SpeechSDK.OutputFormat.Detailed;
        speechConfig.requestWordLevelTimestamps()
        speechConfig.setServiceProperty('punctuation', 'explicit', SpeechSDK.ServicePropertyChannel.UriQueryParameter);

        speechConfig.speechRecognitionLanguage = lang;
        return speechConfig;
    }

    var audioConfig = getAudioConfig();
    var speechConfig = getSpeechConfig()!;
    speechConfig.setProperty(PropertyId.SpeechServiceConnection_AtStartLanguageIdPriority, "Accuracy");


    const autoDetectSourceLanguageConfig = SpeechSDK.AutoDetectSourceLanguageConfig.fromLanguages(["en-US", "fr-FR", "es-ES", lang]);


    const reco = SpeechSDK.SpeechRecognizer.FromConfig(speechConfig, autoDetectSourceLanguageConfig, audioConfig);
    applyCommonConfigurationTo(reco);

    reco.startContinuousRecognitionAsync();

Would it be possible for you to point me in the right direction?
Thanks and happy week-end

Closing comment (2022-10-01T23:35:34Z): Thank you for your answer,
I found the issue. It is related to #1054.
I was expected to get the punctuation on the word level.
I managed to hack a script matching the "Display" field with the words. But it doesn't work in certain cases like "100%" and "one hundred percent"
On this issue, it says
"With the 3.1 version of the API (currently in preview) you can request word-level timestamps on the display form with the displayFormWordLevelTimestampsEnabled property."
Could you point me if and how I can activate that on the js sdk (installed via npm) ?
Thanks