Repository: vscode
Issue: Semantic tokens types do not override "standard token type" and "font style"
Labels: bug,grammar,semantic-tokens

<!-- ⚠️⚠️ Do Not Delete This! bug_report_template ⚠️⚠️ -->
<!-- Please read our Rules of Conduct: https://opensource.microsoft.com/codeofconduct/ -->
<!-- 🕮 Read our guide about submitting issues: https://github.com/microsoft/vscode/wiki/Submitting-Bugs-and-Suggestions -->
<!-- 🔎 Search existing issues to avoid creating duplicates. -->
<!-- 🧪 Test using the latest Insiders build to see if your issue has already been fixed: https://code.visualstudio.com/insiders/ -->
<!-- 💡 Instead of creating your report here, use 'Report Issue' from the 'Help' menu in VS Code to pre-fill useful information. -->
<!-- 🔧 Launch with `code --disable-extensions` to check. -->
Does this issue occur when all extensions are disabled?: Yes

<!-- 🪓 If you answered No above, use 'Help: Start Extension Bisect' from Command Palette to try to identify the cause. -->
<!-- 📣 Issues caused by an extension need to be reported directly to the extension publisher. The 'Help > Report Issue' dialog can assist with this. -->
- VS Code Version: `1.93.1`
- OS Version: Windows 11 Pro

Context: I'm working on a project to automatically generate TextMate grammars (for use in VS Code) from general context-free grammars (where productions can be annotated with scope names). As TextMate grammars are less expressive than CFGs, the generated TextMate grammars sometimes give rise to tokenization mistakes (relative to the original CFG). To alleviate this, there is an additional LSP server with a semantic highlighter (accurate CFG parser) to correct tokenization mistakes. The issue is that not all corrections provided by the semantic highlighter seem to take effect. I've tried to make a minimal example to reproduce the issue, using just the standard semantic highlighter for TypeScript and a small synthetic TextMate grammar for TS (in a separate extension).

Steps to Reproduce:

 1. Create a new extension with the following files:
      
      - `package.json`
      
        ```json
        {
          ...
          "contributes": {
            "languages": [
              {
                "id": "typescript",
                "aliases": ["TypeScript"],
                "extensions": [".ts"]
              }
            ],
            "grammars": [
              {
                "language": "typescript",
                "scopeName": "source.typescript",
                "path": "./syntaxes/typescript.tmLanguage.json"
              }
            ]
          }
        }
        ```
        
      - `syntaxes/typescript.tmLanguage.json`
      
        ```json
        {
          "scopeName": "source.typescript",
          "patterns": [
            {
              "match": "foo",
              "name": "comment"
            }
          ]
        }
        ```
    
 2. Run the extension (with all other extensions disabled)
 
 3. Create the following file `test.ts`:
 
    ```
    const foo = 5;
    ```
 
 4. Set theme `Solarized Light` (or any other theme that italicizes comments)

 5. Open the scope inspector:
 
    ![image](https://github.com/user-attachments/assets/d5b30b62-929f-4b04-8f43-0c1253c99dc9)
    
    **Expected:**
      - "standard token type" is `Other` (I'd expect "semantic token type" to take precedence over the "textmate scope")
      - "font style" is absent (as the strikethrough in the screenshot suggests), so `foo` is not italicized

    **Actual:**
      - "standard token type" is `Comment`
      - "font style" is present and `italic` (if this is the intended behavior, then the strikethrough could be improved to better reflect this), so `foo` is italicized