Repository: autozimu/LanguageClient-neovim (language: rust)

Issue:
Preset LanguageClient_semanticHighlightMaps for known servers
## Is your feature request related to a problem? Please describe.

clangd and rust-analyzer (and maybe others) have semantic highlighting, would be nice if it worked out of the box.

## Describe the solution you'd like

As far as I understand, vim has a number of preset highlight classes which are used by the built-in syntax highlighting. They very likely already look great on the user's theme. The default `semanticHighlightMaps` could map to these "built-in" highlight classes.

## Describe alternatives you've considered

1. Mapping to generic classes like `LanguageClientType`, `LanguageClientMember`, etc.

    This is less work for the user to set up as it's somewhat generic between different language servers, but it's still not out of the box.

## Additional context

I'm not very familiar with vim syntax highlighting, maybe the built-in classes are not really sufficient?



Discussion / hints:
That is a great idea. We probably need to adjust the semantic highlighting implementation first though, as I believe that was a proposal in 3.16 when 3.15 was the current protocol version and it eventually got replaced/renamed by `semantic tokens`, which changes a few detail implementations. So probably a good place to start would be to add support for that again (possibly breaking the feature with servers that are stuck in the old, proposed implementation) and then adding our default mappings.

Fix patch:
diff --git a/autoload/LanguageClient.vim b/autoload/LanguageClient.vim
index c626666f2..0f8c4b0c0 100644
--- a/autoload/LanguageClient.vim
+++ b/autoload/LanguageClient.vim
@@ -896,6 +896,22 @@ function! LanguageClient#Notify(method, params) abort
                 \ }))
 endfunction
 
+function! LanguageClient#textDocument_semanticTokensFull(...) abort
+    if s:ShouldUseFloatWindow() && s:MoveIntoHoverPreview('__LCNHover__')
+        return
+    endif
+    let l:Callback = get(a:000, 1, v:null)
+    let l:params = {
+                \ 'filename': LSP#filename(),
+                \ 'text': LSP#text(),
+                \ 'line': LSP#line(),
+                \ 'character': LSP#character(),
+                \ 'handle': s:IsFalse(l:Callback),
+                \ }
+    call extend(l:params, get(a:000, 0, {}))
+    return LanguageClient#Call('textDocument/semanticTokens/full', l:params, l:Callback)
+endfunction
+
 function! LanguageClient#textDocument_hover(...) abort
     if s:ShouldUseFloatWindow() && s:MoveIntoHoverPreview('__LCNHover__')
         return
@@ -1681,20 +1697,6 @@ function! s:print_semantic_scopes(response) abort
     echo l:msg
 endfunction
 
-function! LanguageClient#showSemanticHighlightSymbols(...) abort
-    let l:params = get(a:000, 0, {})
-    let l:Callback = get(a:000, 1, v:null)
-
-    return LanguageClient#Call('languageClient/showSemanticHighlightSymbols', l:params, l:Callback)
-endfunction
-
-function! LanguageClient_showCursorSemanticHighlightSymbols(...) abort
-    let l:params = get(a:000, 0, {})
-    let l:Callback = get(a:000, 1, function('s:print_cursor_semantic_symbol'))
-
-    return LanguageClient#showSemanticHighlightSymbols(l:params, l:Callback)
-endfunction
-
 function! s:print_cursor_semantic_symbol(response) abort
     let l:symbols = a:response.result
     let l:lines = []
diff --git a/doc/LanguageClient.txt b/doc/LanguageClient.txt
index 0714b059d..903ede954 100644
--- a/doc/LanguageClient.txt
+++ b/doc/LanguageClient.txt
@@ -515,104 +515,7 @@ Valid options: 1 | 0
 
 2.33 g:LanguageClient_semanticHighlightMaps   *g:LanguageClient_semanticHighlightMaps*
 
-String to list/map map. Defines the mapping of semantic highlighting "scopes" to
-highlight groups. This depends on the LSP server supporting the proposed
-semantic highlighting protocol, see:
-
-https://github.com/microsoft/language-server-protocol/issues/18
-https://github.com/microsoft/vscode-languageserver-node/issues/368
-
-
-Like |g:LanguageClient_serverCommands| this is a map where the keys are
-filetypes. However each submap has |regexp| keys and highlight group names
-as values (see |highlight-groups|).
->
-    let g:LanguageClient_semanticHighlightMaps = {
-        \ 'java': {
-        \   '^entity.name.function.java': 'Function',
-        \   '^entity.name.type.class.java': 'Type',
-        \   '^[^:]*entity.name.function.java': 'Function',
-        \   '^[^:]entity.name.type.class.java': 'Type'
-        \ }
-        \ }
-
-The |regexp| in the keys will be used to match semantic scopes. Then any symbols
-that have a semantic scope that matches the key will be highlighted with the
-associated highlight group value. Currently there is no defined order if a
-semantic scope can match multiple keys, so it is recommended to make the keys
-more specific to only match the desired scope(s).
-
-There are a fixed set of semantic scopes defined by the LSP server on startup.
-These can be viewed by calling |LanguageClient_showSemanticScopes| which will
-show all the semantic scopes and their currently mapped highlight group for
-the currently open buffer's filetype.
->
-     call LanguageClient_showSemanticScopes()
-
-     == Output from eclipse.jdt.ls ==
-
-     Highlight Group:
-      None
-     Semantic Scope:
-      invalid.deprecated.java
-       meta.class.java
-        source.java
-
-     Highlight Group:
-      None
-     Semantic Scope:
-      variable.other.autoboxing.java
-       meta.method.body.java
-        meta.method.java
-         meta.class.body.java
-          meta.class.java
-           source.java
-
-     ...
-
-Each semantic scope is a list of strings. They are printed with increasing
-indent to make it easier to read. For example the first scope is:
->
-     ['invalid.deprecated.java', 'meta.class.java', 'source.java']
-
-It is currently isn't mapped to any highlight group as indicated by the None.
-
-Often its more useful to find what semantic scope corresponds to a piece of
-text. This can be done by calling |LanguageClient_showCursorSemanticHighlightSymbols|
-while hovering over the text of interest.
->
-     call LanguageClient_showCursorSemanticHighlightSymbols()
-
-When matching the semantic scopes to keys in |LanguageClient_semanticHighlightMaps|,
-the scopes are concatentated using |LanguageClient_semanticScopeSeparator|
-which is set to the string |':'| by default. For the previous example the
-semantic scope would have this string form using the default separator:
->
-     invalid.deprecated.java:meta.class.java:source.java
-
-Here are a couple of example |regexp| keys that can/cannot match this scope:
->
-     'meta.class.java' =~ 'invalid.deprecated.java:meta.class.java:source.java'
-     '^meta.class.java' !~ 'invalid.deprecated.java:meta.class.java:source.java'
-     '^invalid.deprecated.java' =~ 'invalid.deprecated.java:meta.class.java:source.java'
-     'source.java$' =~ 'invalid.deprecated.java:meta.class.java:source.java'
-     'meta.class.java:source.java' =~ 'invalid.deprecated.java:meta.class.java:source.java'
-     'invalid.deprecated.java:.*:source.java' =~ 'invalid.deprecated.java:meta.class.java:source.java'
-
-Example configuration for eclipse.jdt.ls:
->
-     let g:LanguageClient_semanticHighlightMaps = {}
-     let g:LanguageClient_semanticHighlightMaps['java'] = {
-       \ '^storage.modifier.static.java:entity.name.function.java': 'JavaStaticMemberFunction',
-       \ '^meta.definition.variable.java:meta.class.body.java:meta.class.java': 'JavaMemberVariable',
-       \ '^entity.name.function.java': 'Function',
-       \ '^[^:]*entity.name.function.java': 'Function',
-       \ '^[^:]*entity.name.type.class.java': 'Type',
-       \ }
-
-     highlight! JavaStaticMemberFunction ctermfg=Green cterm=none guifg=Green gui=none
-     highlight! JavaMemberVariable ctermfg=White cterm=italic guifg=White gui=italic
-
+Deprecated, see g:LanguageClient_semanticTokenMappings.
 
 2.34 g:LanguageClient_applyCompletionAdditionalTextEdits *g:LanguageClient_applyCompletionAdditionalTextEdits*
 
@@ -759,6 +662,43 @@ Show completion item documentation in a floating window right next to pmenu.
 Default: 1
 Valid options: 1 | 0
 
+2.46 g:LanguageClient_semanticHighlightingEnabled *g:LanguageClient_semanticHighlightingEnabled*
+
+If enabled, the client will call the LSP semantic tokens method after opening,
+modifying or saving a buffer to try and apply semantic highlighting to the
+buffer.
+
+Default: 0
+Valid options: 1 | 0
+
+2.47 g:LanguageClient_semanticTokenMappings *g:LanguageClient_semanticTokenMappings*
+
+Maps LSP token types to vim highlight groups. Each item in this array is a
+dictionary with the fields name, modifiers and highlightGroup, where name is
+the name of the LSP token type, modifiers is a list of modifiers that need to
+be present in that token for this mapping to match and highlightGroup is the
+vim highlight group that should be applied to the token.
+
+Example:
+
+>
+let g:LanguageClient_semanticTokenMappings = [
+  { 'name': 'comment', 'modifiers': [], 'highlightGroup': 'Comment' },
+  { 'name': 'comment', 'modifiers': ['documentation'], 'highlightGroup': 'Question' },
+];
+<
+
+The first mapping in that example maps the tokens of type `comment` to the
+highlight group `Comment`, and the second matches the tokens of type `comment`
+that have the `documentation` modifier to the `Question` highlight group, this
+way documentation comments and regular comments are (potentially) displayed
+with a different colour.
+
+Default: []
+
+For a full list of the possible token types (name) see the LSP documentation:
+https://microsoft.github.io/language-server-protocol/specifications/specification-current/
+
 ==============================================================================
 3. Commands                                           *LanguageClientCommands*
 
diff --git a/src/config/mod.rs b/src/config/mod.rs
index 5780a80ce..ddf43bbe6 100644
--- a/src/config/mod.rs
+++ b/src/config/mod.rs
@@ -1,7 +1,5 @@
 mod server_command;
 
-pub use server_command::*;
-
 use crate::{
     types::{
         CodeLensDisplay, DiagnosticsDisplay, DiagnosticsList, DocumentHighlightDisplay,
@@ -12,6 +10,7 @@ use crate::{
 use anyhow::{anyhow, Result};
 use lsp_types::{DiagnosticSeverity, MarkupKind, MessageType, TraceOption};
 use serde::Deserialize;
+pub use server_command::*;
 use std::collections::HashMap;
 use std::{path::PathBuf, str::FromStr, time::Duration};
 
@@ -32,6 +31,26 @@ impl LoggerConfig {
     }
 }
 
+#[derive(Debug, Clone, Deserialize, PartialEq)]
+#[serde(rename_all = "camelCase")]
+pub struct SemanticTokenMapping {
+    pub name: String,
+    #[serde(default)]
+    pub modifiers: Vec<String>,
+    pub highlight_group: String,
+}
+
+impl SemanticTokenMapping {
+    #[allow(dead_code)]
+    pub fn new(name: &str, modifiers: &[&str], highlight_group: &str) -> Self {
+        Self {
+            name: name.to_owned(),
+            modifiers: modifiers.iter().map(|i| i.to_string()).collect(),
+            highlight_group: highlight_group.to_owned(),
+        }
+    }
+}
+
 #[derive(Debug)]
 pub struct Config {
     pub auto_start: bool,
@@ -59,22 +78,32 @@ pub struct Config {
     pub selection_ui_auto_open: bool,
     pub use_virtual_text: UseVirtualText,
     pub echo_project_root: bool,
-    pub semantic_highlight_maps: HashMap<String, HashMap<String, String>>,
-    pub semantic_scope_separator: String,
     pub apply_completion_text_edits: bool,
     pub preferred_markup_kind: Option<Vec<MarkupKind>>,
     pub hide_virtual_texts_on_insert: bool,
     pub enable_extensions: Option<HashMap<String, bool>>,
     pub restart_on_crash: bool,
     pub max_restart_retries: u8,
+    /// semantic_token_mappings is a vec of SemanticTokenMappings, where a SemanticTokenMapping
+    /// defines the token type by it's name, the modifiers and the highlight group to be applied to
+    /// it.
+    ///
+    /// If no modifiers are configured for a type it will apply for all tokens of that type.
+    ///
+    /// For example:
+    ///
+    /// [
+    ///     { "name": "function", "modifiers": ["async"], "highlightGroup": "Function" }
+    ///     { "name": "type", "modifiers": [], "highlightGroup": "Type" }
+    /// ]
+    pub semantic_token_mappings: Vec<SemanticTokenMapping>,
+    pub semantic_highlighting_enabled: bool,
 }
 
 impl Default for Config {
     fn default() -> Self {
         Self {
             server_commands: HashMap::new(),
-            semantic_highlight_maps: HashMap::new(),
-            semantic_scope_separator: ":".into(),
             auto_start: true,
             selection_ui: SelectionUI::LocationList,
             selection_ui_auto_open: true,
@@ -105,6 +134,8 @@ impl Default for Config {
             is_nvim: false,
             restart_on_crash: true,
             max_restart_retries: 5,
+            semantic_token_mappings: vec![],
+            semantic_highlighting_enabled: false,
         }
     }
 }
@@ -135,8 +166,6 @@ struct DeserializableConfig {
     selection_ui_auto_open: u8,
     use_virtual_text: UseVirtualText,
     echo_project_root: u8,
-    semantic_highlight_maps: HashMap<String, HashMap<String, String>>,
-    semantic_scope_separator: String,
     apply_completion_text_edits: u8,
     preferred_markup_kind: Option<Vec<MarkupKind>>,
     hide_virtual_texts_on_insert: u8,
@@ -144,6 +173,8 @@ struct DeserializableConfig {
     code_lens_display: Option<CodeLensDisplay>,
     restart_on_crash: u8,
     max_restart_retries: u8,
+    semantic_token_mappings: Vec<SemanticTokenMapping>,
+    semantic_highlighting_enabled: u8,
 }
 
 impl Config {
@@ -172,8 +203,6 @@ impl Config {
             "selection_ui_auto_open": !!s:GetVar('LanguageClient_selectionUI_autoOpen', 1),
             "use_virtual_text": s:useVirtualText(),
             "echo_project_root": !!s:GetVar('LanguageClient_echoProjectRoot', 1),
-            "semantic_highlight_maps": s:GetVar('LanguageClient_semanticHighlightMaps', {}),
-            "semantic_scope_separator": s:GetVar('LanguageClient_semanticScopeSeparator', ':'),
             "apply_completion_text_edits": get(g:, 'LanguageClient_applyCompletionAdditionalTextEdits', 1),
             "preferred_markup_kind": get(g:, 'LanguageClient_preferredMarkupKind', v:null),
             "hide_virtual_texts_on_insert": s:GetVar('LanguageClient_hideVirtualTextsOnInsert', 0),
@@ -182,10 +211,11 @@ impl Config {
             "restart_on_crash": get(g:, 'LanguageClient_restartOnCrash', 1),
             "max_restart_retries": get(g:, 'LanguageClient_maxRestartRetries', 5),
             "server_stderr": get(g:, 'LanguageClient_serverStderr', v:null),
+            "semantic_token_mappings": get(g:, 'LanguageClient_semanticTokenMappings', []),
+            "semantic_highlighting_enabled": get(g:, 'LanguageClient_semanticHighlightingEnabled', 0),
         }"#;
 
         let res: DeserializableConfig = vim.eval(req.replace("\n", ""))?;
-
         let loaded_fzf = vim.eval::<_, i64>("get(g:, 'loaded_fzf')")? == 1;
         let selection_ui = match res.selection_ui {
             Some(s) => SelectionUI::from_str(&s)?,
@@ -238,14 +268,14 @@ impl Config {
             selection_ui_auto_open: res.selection_ui_auto_open == 1,
             use_virtual_text: res.use_virtual_text,
             echo_project_root: res.echo_project_root == 1,
-            semantic_highlight_maps: res.semantic_highlight_maps,
-            semantic_scope_separator: res.semantic_scope_separator,
             apply_completion_text_edits: res.apply_completion_text_edits == 1,
             preferred_markup_kind: res.preferred_markup_kind,
             hide_virtual_texts_on_insert: res.hide_virtual_texts_on_insert == 1,
             enable_extensions: res.enable_extensions,
             restart_on_crash: res.restart_on_crash == 1,
             max_restart_retries: res.max_restart_retries,
+            semantic_token_mappings: res.semantic_token_mappings,
+            semantic_highlighting_enabled: res.semantic_highlighting_enabled == 1,
         })
     }
 }
diff --git a/src/language_server_protocol.rs b/src/language_server_protocol.rs
index a6585e7d9..31cc9b946 100644
--- a/src/language_server_protocol.rs
+++ b/src/language_server_protocol.rs
@@ -1,4 +1,4 @@
-use crate::config::{Config, LoggerConfig, ServerCommand};
+use crate::config::{Config, LoggerConfig, SemanticTokenMapping, ServerCommand};
 use crate::extensions::java;
 use crate::language_client::LanguageClient;
 use crate::sign::Sign;
@@ -7,9 +7,9 @@ use crate::{
     rpcclient::RpcClient,
     types::*,
     utils::{
-        apply_text_edits, code_action_kind_as_str, convert_to_vim_str, decode_parameter_label,
-        escape_single_quote, expand_json_path, get_default_initialization_options, get_root_path,
-        vim_cmd_args_to_value, Canonicalize, Combine, ToUrl,
+        apply_text_edits, code_action_kind_as_str, decode_parameter_label, escape_single_quote,
+        expand_json_path, get_default_initialization_options, get_root_path, vim_cmd_args_to_value,
+        Canonicalize, Combine, ToUrl,
     },
     viewport,
     watcher::FSWatch,
@@ -39,12 +39,14 @@ use lsp_types::{
     ParameterInformation, ParameterInformationSettings, PartialResultParams, Position,
     ProgressParams, ProgressParamsValue, PublishDiagnosticsClientCapabilities,
     PublishDiagnosticsParams, Range, ReferenceContext, RegistrationParams, RenameParams,
-    ResourceOp, SemanticHighlightingClientCapability, SemanticHighlightingParams,
-    ShowMessageParams, ShowMessageRequestParams, SignatureHelp, SignatureHelpClientCapabilities,
-    SignatureInformationSettings, SymbolInformation, TextDocumentClientCapabilities,
-    TextDocumentContentChangeEvent, TextDocumentIdentifier, TextDocumentItem,
-    TextDocumentPositionParams, TextDocumentSyncCapability, TextDocumentSyncKind, TextEdit,
-    UnregistrationParams, VersionedTextDocumentIdentifier, WorkDoneProgress,
+    ResourceOp, SemanticToken, SemanticTokenModifier, SemanticTokensClientCapabilities,
+    SemanticTokensClientCapabilitiesRequests, SemanticTokensFullOptions, SemanticTokensLegend,
+    SemanticTokensParams, SemanticTokensResult, SemanticTokensServerCapabilities,
+    ServerCapabilities, ShowMessageParams, ShowMessageRequestParams, SignatureHelp,
+    SignatureHelpClientCapabilities, SignatureInformationSettings, SymbolInformation,
+    TextDocumentClientCapabilities, TextDocumentContentChangeEvent, TextDocumentIdentifier,
+    TextDocumentItem, TextDocumentPositionParams, TextDocumentSyncCapability, TextDocumentSyncKind,
+    TextEdit, UnregistrationParams, VersionedTextDocumentIdentifier, WorkDoneProgress,
     WorkDoneProgressParams, WorkspaceClientCapabilities, WorkspaceEdit, WorkspaceSymbolParams,
 };
 use maplit::hashmap;
@@ -108,8 +110,6 @@ impl LanguageClient {
             log::error!("{}", err);
         }
         let mut config = config?;
-        let semantic_highlight_language_ids: Vec<String> =
-            config.semantic_highlight_maps.keys().cloned().collect();
 
         // merge defaults with user provided config
         let mut diagnostics_display = self.get_config(|c| c.diagnostics_display.clone())?;
@@ -124,15 +124,11 @@ impl LanguageClient {
 
         self.update_config(|c| *c = config)?;
 
-        self.update_state(|state| {
-            state.semantic_scope_to_hl_group_table.clear();
+        // self.update_state(|state| {
+        //     state.semantic_scope_to_hl_group_table.clear();
 
-            Ok(())
-        })?;
-
-        for language_id in semantic_highlight_language_ids {
-            self.update_semantic_highlight_tables(&language_id)?;
-        }
+        //     Ok(())
+        // })?;
 
         Ok(())
     }
@@ -670,75 +666,6 @@ impl LanguageClient {
         Ok(())
     }
 
-    #[tracing::instrument(level = "info", skip(self))]
-    fn parse_semantic_scopes(&self, language_id: &str, result: &Value) -> Result<()> {
-        let result = InitializeResult::deserialize(result)?;
-
-        if let Some(capability) = result.capabilities.semantic_highlighting {
-            self.update_state(|state| {
-                state
-                    .semantic_scopes
-                    .insert(language_id.into(), capability.scopes.unwrap_or_default());
-                Ok(())
-            })?;
-        }
-
-        Ok(())
-    }
-
-    /// Build the Semantic Highlight Lookup Table of
-    ///
-    /// ScopeIndex -> Option<HighlightGroup>
-    #[tracing::instrument(level = "info", skip(self))]
-    fn update_semantic_highlight_tables(&self, language_id: &str) -> Result<()> {
-        let opt_scopes = self.get_state(|state| state.semantic_scopes.get(language_id).cloned())?;
-        let opt_hl_map =
-            self.get_config(|c| c.semantic_highlight_maps.get(language_id).cloned())?;
-        let scope_separator = self.get_config(|c| c.semantic_scope_separator.clone())?;
-        if let (Some(semantic_scopes), Some(shm)) = (opt_scopes, opt_hl_map) {
-            let mut table: Vec<Option<String>> = Vec::new();
-
-            for scope_list in semantic_scopes {
-                // Combine all scopes ["scopeA", "scopeB", ...] -> "scopeA:scopeB:..."
-                let scope_str = scope_list.iter().join(&scope_separator);
-
-                let mut matched = false;
-                for (scope_regex, hl_group) in &shm {
-                    let match_expr = format!(
-                        "({} =~ {})",
-                        convert_to_vim_str(&scope_str),
-                        convert_to_vim_str(scope_regex)
-                    );
-
-                    let matches: i32 = self.vim()?.eval(match_expr)?;
-
-                    if matches == 1 {
-                        table.push(Some(hl_group.clone()));
-                        matched = true;
-                        break;
-                    }
-                }
-
-                if !matched {
-                    table.push(None);
-                }
-            }
-
-            self.update_state(|state| {
-                state
-                    .semantic_scope_to_hl_group_table
-                    .insert(language_id.into(), table);
-                Ok(())
-            })?;
-        } else {
-            self.update_state(|state| {
-                state.semantic_scope_to_hl_group_table.remove(language_id);
-                Ok(())
-            })?;
-        }
-        Ok(())
-    }
-
     pub fn get_line(&self, path: impl AsRef<Path>, line: u32) -> Result<String> {
         let value: Value = self.vim()?.rpcclient.call(
             "getbufline",
@@ -902,7 +829,7 @@ impl LanguageClient {
             let _ = self.vim()?.echoerr("You seem to be using an incorrect workspace settings format for LanguageClient-neovim, to learn more about this error see `:help g:LanguageClient_settingsPath`");
         }
 
-        let initialization_options = merged_initialization_options(&command, &settings)?;
+        let initialization_options = merged_initialization_options(&command, &settings);
 
         let result: Value = self.get_client(&Some(language_id.clone()))?.call(
             lsp_types::request::Initialize::METHOD,
@@ -987,11 +914,19 @@ impl LanguageClient {
                         code_lens: Some(CodeLensClientCapabilities {
                             dynamic_registration: Some(true),
                         }),
-                        semantic_highlighting_capabilities: Some(
-                            SemanticHighlightingClientCapability {
-                                semantic_highlighting: true,
+                        semantic_tokens: Some(SemanticTokensClientCapabilities {
+                            dynamic_registration: None,
+                            requests: SemanticTokensClientCapabilitiesRequests {
+                                range: Some(false),
+                                full: Some(SemanticTokensFullOptions::Bool(true)),
                             },
-                        ),
+                            token_types: vec![],
+                            token_modifiers: vec![],
+                            formats: vec![],
+                            overlapping_token_support: Some(false),
+                            multiline_token_support: Some(false),
+                        }),
+                        semantic_highlighting_capabilities: None,
                         hover: Some(HoverClientCapabilities {
                             content_format: preferred_markup_kind,
                             ..HoverClientCapabilities::default()
@@ -1026,6 +961,19 @@ impl LanguageClient {
                     .combine(&json!({ name: options }));
             }
 
+            let capabilities: ServerCapabilities = initialize_result.capabilities.clone();
+            if let Some(cap) = capabilities.semantic_tokens_provider {
+                let legend = match cap {
+                    SemanticTokensServerCapabilities::SemanticTokensOptions(c) => c.legend,
+                    SemanticTokensServerCapabilities::SemanticTokensRegistrationOptions(c) => {
+                        c.semantic_tokens_options.legend
+                    }
+                };
+                state
+                    .semantic_token_legends
+                    .insert(language_id.clone(), legend);
+            }
+
             state
                 .capabilities
                 .insert(language_id.clone(), initialize_result);
@@ -1043,11 +991,6 @@ impl LanguageClient {
             error!("{}\n{:?}", message, e);
             self.vim()?.echoerr(&message)?;
         }
-        if let Err(e) = self.parse_semantic_scopes(&language_id, &result) {
-            let message = format!("LanguageClient: failed to parse semantic scopes: {}", e);
-            error!("{}\n{:?}", message, e);
-            self.vim()?.echoerr(&message)?;
-        }
 
         Ok(result)
     }
@@ -1056,7 +999,6 @@ impl LanguageClient {
     fn initialized(&self, params: &Value) -> Result<()> {
         let filename = self.vim()?.get_filename(params)?;
         let language_id = self.vim()?.get_language_id(&filename, params)?;
-        self.update_semantic_highlight_tables(&language_id)?;
         self.get_client(&Some(language_id))?.notify(
             lsp_types::notification::Initialized::METHOD,
             InitializedParams {},
@@ -2289,214 +2231,57 @@ impl LanguageClient {
     }
 
     #[tracing::instrument(level = "info", skip(self))]
-    pub fn text_document_semantic_highlight(&self, params: &Value) -> Result<()> {
-        let mut params = SemanticHighlightingParams::deserialize(params)?;
-
-        // TODO: Do we need to handle the versioning of the file?
-        let mut filename = params
-            .text_document
-            .uri
-            .filepath()?
-            .to_string_lossy()
-            .into_owned();
-        // Workaround bug: remove first '/' in case of '/C:/blabla'.
-        if filename.starts_with('/') && filename.chars().nth(2) == Some(':') {
-            filename.remove(0);
+    pub fn text_document_semantic_tokens_full(&self, params: &Value) -> Result<Value> {
+        let is_enabled = self
+            .get_config(|c| c.semantic_highlighting_enabled)
+            .unwrap_or_default();
+        if !is_enabled {
+            return Ok(Value::Null);
         }
-        // Unify name to avoid mismatch due to case insensitivity.
-        let filename = filename.canonicalize();
-        let language_id = self.vim()?.get_language_id(&filename, &Value::Null)?;
-
-        let opt_hl_table = self.get_state(|state| {
-            state
-                .semantic_scope_to_hl_group_table
-                .get(&language_id)
-                .cloned()
-        })?;
 
-        // Sort lines in ascending order
-        params.lines.sort_by(|a, b| a.line.cmp(&b.line));
+        let filename = self.vim()?.get_filename(params)?;
+        let language_id = self.vim()?.get_language_id(&filename, params)?;
+        let client = self.get_client(&Some(language_id.clone()))?;
+        let response: SemanticTokensResult = client.call(
+            lsp_types::request::SemanticTokensFullRequest::METHOD,
+            SemanticTokensParams {
+                text_document: TextDocumentIdentifier {
+                    uri: filename.to_url()?,
+                },
+                work_done_progress_params: WorkDoneProgressParams::default(),
+                partial_result_params: PartialResultParams::default(),
+            },
+        )?;
 
-        // Remove obviously invalid values
-        while let Some(line_info) = params.lines.first() {
-            if line_info.line >= 0 {
-                break;
-            } else {
-                warn!(
-                    "Invalid Semantic Highlight Line: {}",
-                    params.lines.remove(0).line
-                );
-            }
+        let legend = self.get_state(|s| s.semantic_token_legends.get(&language_id).cloned())?;
+        if legend.is_none() {
+            return Ok(Value::Null);
         }
+        let legend = legend.unwrap();
 
-        let semantic_hl_state = TextDocumentSemanticHighlightState {
-            last_version: params.text_document.version,
-            symbols: params.lines,
-            highlights: None,
-        };
-
-        if let Some(hl_table) = opt_hl_table {
-            let ns_id = self.get_or_create_namespace(&LCNamespace::SemanticHighlight)?;
-
-            let buffer = self.vim()?.get_bufnr(&filename, &Value::Null)?;
-
-            if buffer == -1 {
-                error!(
-                    "Received Semantic Highlighting for non-open buffer: {}",
-                    filename
-                );
-                return Ok(());
-            }
-
-            /*
-             * Currently servers update entire regions of text at a time or a
-             * single line so simply clear between the first and last line to
-             * ensure no highlights are left dangling
-             */
-            let mut clear_region: Option<(u32, u32)> = None;
-            let mut highlights = Vec::with_capacity(semantic_hl_state.symbols.len());
-
-            for line in &semantic_hl_state.symbols {
-                if let Some(tokens) = &line.tokens {
-                    for token in tokens {
-                        if token.length == 0 {
-                            continue;
-                        }
-
-                        if let Some(Some(group)) = hl_table.get(token.scope as usize) {
-                            highlights.push(Highlight {
-                                line: line.line as u32,
-                                character_start: token.character,
-                                character_end: token.character + token.length as u32,
-                                group: group.clone(),
-                                text: String::new(),
-                            });
-                        }
-                    }
-
-                    match clear_region {
-                        Some((begin, _)) => {
-                            clear_region = Some((begin, line.line as u32 + 1));
-                        }
-                        None => {
-                            clear_region = Some((line.line as u32, line.line as u32 + 1));
-                        }
-                    }
-                }
-            }
-
-            info!(
-                "Semantic Highlighting Region [{}, {}]:",
-                semantic_hl_state
-                    .symbols
-                    .first()
-                    .map_or(-1, |h| h.line as i64),
-                semantic_hl_state
-                    .symbols
-                    .last()
-                    .map_or(-1, |h| h.line as i64)
-            );
-
-            info!(
-                "Semantic Highlighting Region (Parsed) [{}, {}]:",
-                highlights.first().map_or(-1, |h| h.line as i64),
-                highlights.last().map_or(-1, |h| h.line as i64)
-            );
+        let mappings = self
+            .get_config(|c| c.semantic_token_mappings.clone())
+            .unwrap_or_default();
+        if mappings.is_empty() {
+            return Ok(Value::Null);
+        }
 
-            let mut clears = Vec::new();
-            if let Some((begin, end)) = clear_region {
-                clears.push(ClearNamespace {
-                    line_start: begin,
-                    line_end: end,
-                });
+        let highlights: Vec<Highlight> = match response {
+            SemanticTokensResult::Tokens(r) => {
+                semantic_tokens_to_highlights(&legend, mappings, r.data)
             }
+            SemanticTokensResult::Partial(_) => vec![],
+        };
 
-            let mut num_semantic_hls = 0;
-            let num_new_semantic_hls = highlights.len();
-
-            self.update_state(|state| {
-                state.vim.rpcclient.notify(
-                    "s:ApplySemanticHighlights",
-                    json!([buffer, ns_id, clears, highlights]),
-                )?;
-
-                let old_semantic_hl_state = state
-                    .semantic_highlights
-                    .insert(language_id.clone(), semantic_hl_state);
-
-                let semantic_hl_state = state.semantic_highlights.get_mut(&language_id).unwrap();
-
-                let mut combined_hls = Vec::with_capacity(highlights.len());
-
-                let mut existing_hls = old_semantic_hl_state
-                    .map_or(Vec::new(), |hl_state| {
-                        hl_state.highlights.unwrap_or_default()
-                    })
-                    .into_iter()
-                    .peekable();
-
-                let mut new_hls = highlights.into_iter().peekable();
-
-                // Incrementally update the highlighting
-                loop {
-                    match (existing_hls.peek(), new_hls.peek()) {
-                        (Some(existing_hl), Some(new_hl)) => {
-                            use std::cmp::Ordering;
-
-                            match existing_hl.line.cmp(&new_hl.line) {
-                                Ordering::Less => {
-                                    if clear_region.unwrap_or((0, 0)).0 <= existing_hl.line
-                                        && existing_hl.line < clear_region.unwrap_or((0, 0)).1
-                                    {
-                                        // within clear region, this highlight gets cleared
-                                        existing_hls.next();
-                                    } else {
-                                        combined_hls
-                                            .push(existing_hls.next().expect("unreachable"));
-                                    }
-                                }
-                                Ordering::Greater => {
-                                    combined_hls.push(new_hls.next().expect("unreachable"));
-                                }
-                                Ordering::Equal => {
-                                    // existing highlight on same line as new, it gets cleared
-                                    existing_hls.next();
-                                }
-                            }
-                        }
-                        (Some(_), None) => {
-                            combined_hls.push(existing_hls.next().expect("unreachable"));
-                        }
-                        (None, Some(_)) => {
-                            combined_hls.push(new_hls.next().expect("unreachable"));
-                        }
-                        (None, None) => {
-                            break;
-                        }
-                    }
-                }
-
-                num_semantic_hls = combined_hls.len();
-
-                semantic_hl_state.highlights = Some(combined_hls);
-
-                Ok(())
-            })?;
-
-            info!(
-                "Applied Semantic Highlighting for {} Symbols ({} new)",
-                num_semantic_hls, num_new_semantic_hls
-            )
-        } else {
-            self.update_state(|state| {
-                state
-                    .semantic_highlights
-                    .insert(language_id.clone(), semantic_hl_state);
-                Ok(())
-            })?;
-        }
+        let ns_id = self.get_or_create_namespace(&LCNamespace::SemanticHighlight)?;
+        let buffer = self.vim()?.get_bufnr(&filename, &Value::Null)?;
+        let clears: Vec<ClearNamespace> = Vec::new();
+        self.vim()?.rpcclient.notify(
+            "s:ApplySemanticHighlights",
+            json!([buffer, ns_id, clears, highlights]),
+        )?;
 
-        Ok(())
+        Ok(Value::Null)
     }
 
     // logs a message to with the specified level to the log file if the threshold is below the
@@ -3336,80 +3121,6 @@ impl LanguageClient {
         Ok(())
     }
 
-    pub fn semantic_scopes(&self, params: &Value) -> Result<Value> {
-        let filename = self.vim()?.get_filename(params)?;
-        let language_id = self.vim()?.get_language_id(&filename, params)?;
-
-        let (scopes, mut scope_mapping) = self.get_state(|state| {
-            (
-                state
-                    .semantic_scopes
-                    .get(&language_id)
-                    .cloned()
-                    .unwrap_or_default(),
-                state
-                    .semantic_scope_to_hl_group_table
-                    .get(&language_id)
-                    .cloned()
-                    .unwrap_or_default(),
-            )
-        })?;
-
-        let mut semantic_scopes = Vec::new();
-
-        // If the user has not set up highlighting yet the table does not exist
-        if scopes.len() > scope_mapping.len() {
-            scope_mapping.resize(scopes.len(), None);
-        }
-
-        for (scope, opt_hl_group) in scopes.iter().zip(scope_mapping.iter()) {
-            if let Some(hl_group) = opt_hl_group {
-                semantic_scopes.push(json!({
-                    "scope": scope,
-                    "hl_group": hl_group,
-                }));
-            } else {
-                semantic_scopes.push(json!({
-                    "scope": scope,
-                    "hl_group": "None",
-                }));
-            }
-        }
-
-        Ok(json!(semantic_scopes))
-    }
-
-    pub fn semantic_highlight_symbols(&self, params: &Value) -> Result<Value> {
-        let filename = self.vim()?.get_filename(params)?;
-        let language_id = self.vim()?.get_language_id(&filename, params)?;
-
-        let (opt_scopes, opt_hl_state) = self.get_state(|state| {
-            (
-                state.semantic_scopes.get(&language_id).cloned(),
-                state.semantic_highlights.get(&language_id).cloned(),
-            )
-        })?;
-
-        if let (Some(scopes), Some(hl_state)) = (opt_scopes, opt_hl_state) {
-            let mut symbols = Vec::new();
-
-            for sym in hl_state.symbols {
-                for token in sym.tokens.unwrap_or_default() {
-                    symbols.push(json!({
-                        "line": sym.line as u64,
-                        "character_start": token.character as u64,
-                        "character_end": token.character as u64 + token.length as u64,
-                        "scope": scopes.get(token.scope as usize).cloned().unwrap_or_default()
-                    }));
-                }
-            }
-
-            Ok(json!(symbols))
-        } else {
-            Ok(json!([]))
-        }
-    }
-
     #[tracing::instrument(level = "info", skip(self))]
     pub fn ncm_refresh(&self, params: &Value) -> Result<Value> {
         let params = NCMRefreshParams::deserialize(params)?;
@@ -4016,10 +3727,7 @@ impl LanguageClient {
     }
 }
 
-fn merged_initialization_options(
-    command: &ServerCommand,
-    settings: &Value,
-) -> Result<Option<Value>> {
+fn merged_initialization_options(command: &ServerCommand, settings: &Value) -> Option<Value> {
     let server_name = command.name();
     let section = format!("/{}", server_name);
     let default_initialization_options = get_default_initialization_options(&server_name);
@@ -4031,16 +3739,223 @@ fn merged_initialization_options(
         .combine(workspace_initialization_options);
 
     if initialization_options.is_null() {
-        Ok(None)
+        None
     } else {
-        Ok(Some(initialization_options))
+        Some(initialization_options)
+    }
+}
+
+fn semantic_tokens_to_highlights(
+    legend: &SemanticTokensLegend,
+    mappings: Vec<SemanticTokenMapping>,
+    tokens: Vec<SemanticToken>,
+) -> Vec<Highlight> {
+    tokens
+        .iter()
+        .enumerate()
+        .filter_map(|(idx, d)| {
+            let token_modifiers = resolve_token_modifiers(&legend, d.token_modifiers_bitset);
+            let mapping = resolve_token_mapping(&legend, &mappings, &token_modifiers, d);
+            let line: u32 = tokens.iter().take(idx + 1).map(|i| i.delta_line).sum();
+            let mut character_start = d.delta_start;
+            // loop backwards over the results for as long as we find that the previous token also
+            // has a delta_line of zero.
+            if idx > 0 && d.delta_line == 0 {
+                let mut inner_idx = idx;
+                loop {
+                    character_start += tokens[inner_idx - 1].delta_start;
+                    if inner_idx - 1 == 0 || tokens[inner_idx - 1].delta_line != 0 {
+                        break;
+                    }
+                    inner_idx -= 1;
+                }
+            }
+            mapping.map(|m| Highlight {
+                line,
+                character_start,
+                character_end: character_start + d.length,
+                group: m.highlight_group.clone(),
+                text: m.highlight_group,
+            })
+        })
+        .collect()
+}
+
+/// LSP uses a bitset to indicate which token modifiers apply for a given token. The actual
+/// modifiers are obtained from the legend at the indexes indicated by the bits set to 1 in the
+/// bitset.
+fn resolve_token_modifiers(
+    legend: &SemanticTokensLegend,
+    bitset: u32,
+) -> Vec<SemanticTokenModifier> {
+    format!("{:#b}", bitset)
+        .replace("0b", "")
+        .chars()
+        .into_iter()
+        .rev()
+        .enumerate()
+        .filter_map(|(idx, c)| match c.to_string().parse::<usize>().unwrap() {
+            0 => None,
+            _ => Some(legend.token_modifiers[idx].clone()),
+        })
+        .collect()
+}
+
+/// LSP uses integers to indicate an encoded semantic token type that corresponds to each token, in
+/// order to get the name of the type we must index the token_types field of the legend with the
+/// number that represents the encoded token type.
+fn resolve_token_mapping(
+    legend: &SemanticTokensLegend,
+    mappings: &[SemanticTokenMapping],
+    token_modifiers: &[SemanticTokenModifier],
+    token: &SemanticToken,
+) -> Option<SemanticTokenMapping> {
+    let token_type = legend.token_types[token.token_type as usize].clone();
+    let mappings: Vec<SemanticTokenMapping> = mappings
+        .iter()
+        .filter(|i| i.name == token_type.as_str())
+        .cloned()
+        .collect();
+    if mappings.is_empty() {
+        return None;
     }
+
+    // get either the mapping that matches both type and modifiers
+    let modifiers: Vec<&str> = token_modifiers.iter().map(|i| i.as_str()).collect();
+    mappings.iter().find(|m| modifiers == m.modifiers).cloned()
 }
 
 #[cfg(test)]
 mod test {
     use super::*;
     use crate::config::{ServerCommand, ServerDetails};
+    use lsp_types::SemanticTokenType;
+
+    #[test]
+    fn it_resolves_the_correct_mapping() {
+        let legend = SemanticTokensLegend {
+            token_types: vec![SemanticTokenType::CLASS, SemanticTokenType::FUNCTION],
+            token_modifiers: vec![],
+        };
+        let mappings = vec![
+            SemanticTokenMapping::new("function", &[], "Function"),
+            SemanticTokenMapping::new("comment", &[], "Comment"),
+        ];
+        let actual = resolve_token_mapping(
+            &legend,
+            &mappings,
+            &[],
+            &SemanticToken {
+                token_type: 1,
+                delta_line: 0,
+                delta_start: 0,
+                length: 0,
+                token_modifiers_bitset: 0,
+            },
+        );
+        let expect = SemanticTokenMapping::new("function", &[], "Function");
+
+        assert!(actual.is_some());
+        assert_eq!(expect, actual.unwrap());
+    }
+
+    #[test]
+    fn it_resolves_to_the_correct_modifiers() {
+        let legend = SemanticTokensLegend {
+            token_types: vec![],
+            token_modifiers: vec![
+                SemanticTokenModifier::DEPRECATED,
+                SemanticTokenModifier::DECLARATION,
+                SemanticTokenModifier::DOCUMENTATION,
+                SemanticTokenModifier::ASYNC,
+            ],
+        };
+
+        // if the token_modifier is 3 then it translates to a 0b0011 bitset, which means we want
+        // the modifiers in index 0 and 1 of the legend.
+        let actual = resolve_token_modifiers(&legend, 3);
+        let expect = vec![
+            SemanticTokenModifier::DEPRECATED,
+            SemanticTokenModifier::DECLARATION,
+        ];
+        assert_eq!(expect, actual);
+
+        // if the token_modifier is 2 then it translates to a 0b0010 bitset, which means we want
+        // the modifiers in index 1 of the legend.
+        let actual = resolve_token_modifiers(&legend, 2);
+        let expect = vec![SemanticTokenModifier::DECLARATION];
+        assert_eq!(expect, actual);
+    }
+
+    #[test]
+    fn it_maps_the_example_in_the_spec() {
+        // inputs:
+        //      { deltaLine: 2, deltaStartChar: 5, length: 3, tokenType: 0, tokenModifiers: 3 },
+        //      { deltaLine: 0, deltaStartChar: 5, length: 4, tokenType: 1, tokenModifiers: 0 },
+        //      { deltaLine: 3, deltaStartChar: 2, length: 7, tokenType: 2, tokenModifiers: 0 }
+        //
+        // outputs:
+        //      { line: 2, startChar:  5, length: 3, tokenType: 0, tokenModifiers: 3 },
+        //      { line: 2, startChar: 10, length: 4, tokenType: 1, tokenModifiers: 0 },
+        //      { line: 5, startChar:  2, length: 7, tokenType: 2, tokenModifiers: 0 }
+        //
+        let tokens = vec![
+            SemanticToken {
+                delta_line: 2,
+                delta_start: 5,
+                length: 3,
+                token_type: 1,
+                token_modifiers_bitset: 0,
+            },
+            SemanticToken {
+                delta_line: 0,
+                delta_start: 5,
+                length: 4,
+                token_type: 1,
+                token_modifiers_bitset: 0,
+            },
+            SemanticToken {
+                delta_line: 3,
+                delta_start: 2,
+                length: 7,
+                token_type: 1,
+                token_modifiers_bitset: 0,
+            },
+        ];
+        let legend = SemanticTokensLegend {
+            token_types: vec![SemanticTokenType::FUNCTION, SemanticTokenType::TYPE],
+            token_modifiers: vec![],
+        };
+        let mappings = vec![
+            SemanticTokenMapping::new("function", &[], "Function"),
+            SemanticTokenMapping::new("type", &[], "Type"),
+        ];
+        let actual = semantic_tokens_to_highlights(&legend, mappings, tokens);
+        let expect = vec![
+            Highlight {
+                line: 2,
+                character_start: 5,
+                character_end: 8,
+                group: "Type".into(),
+                text: "Type".into(),
+            },
+            Highlight {
+                line: 2,
+                character_start: 10,
+                character_end: 14,
+                group: "Type".into(),
+                text: "Type".into(),
+            },
+            Highlight {
+                line: 5,
+                character_start: 2,
+                character_end: 9,
+                group: "Type".into(),
+                text: "Type".into(),
+            },
+        ];
+        assert_eq!(expect, actual);
+    }
 
     #[test]
     fn test_expands_initialization_options() {
@@ -4063,8 +3978,7 @@ mod test {
             handlers: None,
         });
 
-        let options = merged_initialization_options(&command, &settings)
-            .expect("could not get initialization options");
+        let options = merged_initialization_options(&command, &settings);
         assert!(options.is_some());
         assert_eq!(
             json!({
@@ -4096,8 +4010,7 @@ mod test {
             handlers: None,
         });
 
-        let options = merged_initialization_options(&command, &settings)
-            .expect("could not get initialization options");
+        let options = merged_initialization_options(&command, &settings);
         assert!(options.is_some());
         assert_eq!(
             json!({
@@ -4123,8 +4036,7 @@ mod test {
             handlers: None,
         });
 
-        let options = merged_initialization_options(&command, &settings)
-            .expect("could not get initialization options");
+        let options = merged_initialization_options(&command, &settings);
         assert!(options.is_some());
         assert_eq!(
             json!({
@@ -4144,8 +4056,7 @@ mod test {
         });
         let command = ServerCommand::Simple(vec!["gopls".into()]);
 
-        let options = merged_initialization_options(&command, &settings)
-            .expect("could not get initialization options");
+        let options = merged_initialization_options(&command, &settings);
         assert!(options.is_some());
         assert_eq!(
             json!({
diff --git a/src/rpchandler.rs b/src/rpchandler.rs
index 00c5f5708..e0ea1c166 100644
--- a/src/rpchandler.rs
+++ b/src/rpchandler.rs
@@ -96,6 +96,9 @@ impl LanguageClient {
             request::ResolveCompletionItem::METHOD => self.completion_item_resolve(&params),
             request::ExecuteCommand::METHOD => self.workspace_execute_command(&params),
             request::ApplyWorkspaceEdit::METHOD => self.workspace_apply_edit(&params),
+            request::SemanticTokensFullRequest::METHOD => {
+                self.text_document_semantic_tokens_full(&params)
+            }
             request::Shutdown::METHOD => self.shutdown(&params),
             request::DocumentHighlightRequest::METHOD => {
                 self.text_document_document_highlight(&params)
@@ -116,8 +119,6 @@ impl LanguageClient {
             REQUEST_CLASS_FILE_CONTENTS => self.java_class_file_contents(&params),
             REQUEST_DEBUG_INFO => self.debug_info(&params),
             REQUEST_CODE_LENS_ACTION => self.handle_code_lens_action(&params),
-            REQUEST_SEMANTIC_SCOPES => self.semantic_scopes(&params),
-            REQUEST_SHOW_SEMANTIC_HL_SYMBOLS => self.semantic_highlight_symbols(&params),
             REQUEST_EXECUTE_CODE_ACTION => self.execute_code_action(&params),
 
             clangd::request::SwitchSourceHeader::METHOD => {
@@ -181,28 +182,38 @@ impl LanguageClient {
             notification::DidChangeConfiguration::METHOD => {
                 self.workspace_did_change_configuration(&params)?
             }
-            notification::DidOpenTextDocument::METHOD => self.text_document_did_open(&params)?,
+            notification::DidOpenTextDocument::METHOD => {
+                self.text_document_did_open(&params)?;
+            }
             notification::DidChangeTextDocument::METHOD => {
-                self.text_document_did_change(&params)?
+                self.text_document_did_change(&params)?;
+            }
+            notification::DidSaveTextDocument::METHOD => {
+                self.text_document_did_save(&params)?;
             }
-            notification::DidSaveTextDocument::METHOD => self.text_document_did_save(&params)?,
             notification::DidCloseTextDocument::METHOD => self.text_document_did_close(&params)?,
             notification::PublishDiagnostics::METHOD => {
                 self.text_document_publish_diagnostics(&params)?
             }
-            notification::SemanticHighlighting::METHOD => {
-                self.text_document_semantic_highlight(&params)?
-            }
             notification::Progress::METHOD => self.progress(&params)?,
             notification::LogMessage::METHOD => self.window_log_message(&params)?,
             notification::ShowMessage::METHOD => self.window_show_message(&params)?,
             notification::Exit::METHOD => self.exit(&params)?,
             // Extensions.
-            NOTIFICATION_HANDLE_FILE_TYPE => self.handle_file_type(&params)?,
+            NOTIFICATION_HANDLE_FILE_TYPE => {
+                self.handle_file_type(&params)?;
+                self.text_document_semantic_tokens_full(&params)?;
+            }
             NOTIFICATION_HANDLE_BUF_NEW_FILE => self.handle_buf_new_file(&params)?,
-            NOTIFICATION_HANDLE_BUF_ENTER => self.handle_buf_enter(&params)?,
+            NOTIFICATION_HANDLE_BUF_ENTER => {
+                self.handle_buf_enter(&params)?;
+                self.text_document_semantic_tokens_full(&params)?;
+            }
             NOTIFICATION_HANDLE_TEXT_CHANGED => self.handle_text_changed(&params)?,
-            NOTIFICATION_HANDLE_BUF_WRITE_POST => self.handle_buf_write_post(&params)?,
+            NOTIFICATION_HANDLE_BUF_WRITE_POST => {
+                self.handle_buf_write_post(&params)?;
+                self.text_document_semantic_tokens_full(&params)?;
+            }
             NOTIFICATION_HANDLE_BUF_DELETE => self.handle_buf_delete(&params)?,
             NOTIFICATION_HANDLE_CURSOR_MOVED => self.handle_cursor_moved(&params, false)?,
             NOTIFICATION_HANDLE_COMPLETE_DONE => self.handle_complete_done(&params)?,
diff --git a/src/types.rs b/src/types.rs
index 25ea1a90e..ead618014 100644
--- a/src/types.rs
+++ b/src/types.rs
@@ -10,7 +10,6 @@ use crate::{viewport::Viewport, vim::Highlight};
 use anyhow::{anyhow, Result};
 use jsonrpc_core::Params;
 use log::*;
-use lsp_types::Range;
 use lsp_types::{
     CodeAction, CodeLens, Command, CompletionItem, CompletionTextEdit, Diagnostic,
     DiagnosticSeverity, DocumentHighlightKind, FileChangeType, FileEvent, Hover, HoverContents,
@@ -18,6 +17,7 @@ use lsp_types::{
     MessageType, NumberOrString, Registration, SemanticHighlightingInformation, SymbolInformation,
     TextDocumentItem, TextDocumentPositionParams, Url, WorkspaceEdit,
 };
+use lsp_types::{Range, SemanticTokensLegend};
 use maplit::hashmap;
 use pathdiff::diff_paths;
 use serde::{Deserialize, Serialize};
@@ -62,8 +62,6 @@ pub const REQUEST_EXPLAIN_ERROR_AT_POINT: &str = "languageClient/explainErrorAtP
 pub const REQUEST_FIND_LOCATIONS: &str = "languageClient/findLocations";
 pub const REQUEST_DEBUG_INFO: &str = "languageClient/debugInfo";
 pub const REQUEST_CODE_LENS_ACTION: &str = "LanguageClient/handleCodeLensAction";
-pub const REQUEST_SEMANTIC_SCOPES: &str = "languageClient/semanticScopes";
-pub const REQUEST_SHOW_SEMANTIC_HL_SYMBOLS: &str = "languageClient/showSemanticHighlightSymbols";
 pub const REQUEST_CLASS_FILE_CONTENTS: &str = "java/classFileContents";
 pub const REQUEST_EXECUTE_CODE_ACTION: &str = "languageClient/executeCodeAction";
 
@@ -156,8 +154,15 @@ pub struct State {
     pub text_documents: HashMap<String, TextDocumentItem>,
     pub viewports: HashMap<String, Viewport>,
     pub text_documents_metadata: HashMap<String, TextDocumentItemMetadata>,
-    pub semantic_scopes: HashMap<String, Vec<Vec<String>>>,
-    pub semantic_scope_to_hl_group_table: HashMap<String, Vec<Option<String>>>,
+    /// semantic_token_legends is a map of language id to semantic tokens legend. The semantic
+    /// tokengs legend is returned by the server upon initialization.
+    ///
+    /// For example:
+    ///
+    /// {
+    ///     "rust": { "token_types": ["type", "struct", "variable"] }
+    /// }
+    pub semantic_token_legends: HashMap<String, SemanticTokensLegend>,
     // filename => semantic highlight state
     pub semantic_highlights: HashMap<String, TextDocumentSemanticHighlightState>,
     // filename => diagnostics.
@@ -226,8 +231,7 @@ impl State {
             text_documents: HashMap::new(),
             viewports: HashMap::new(),
             text_documents_metadata: HashMap::new(),
-            semantic_scopes: HashMap::new(),
-            semantic_scope_to_hl_group_table: HashMap::new(),
+            semantic_token_legends: HashMap::new(),
             semantic_highlights: HashMap::new(),
             inlay_hints: HashMap::new(),
             code_lens: HashMap::new(),
@@ -679,14 +683,14 @@ impl ToRpcError for anyhow::Error {
 }
 
 pub trait ToParams {
-    fn to_params(self) -> Result<Params>;
+    fn to_params(&self) -> Result<Params>;
 }
 
 impl<T> ToParams for T
 where
     T: Serialize,
 {
-    fn to_params(self) -> Result<Params> {
+    fn to_params(&self) -> Result<Params> {
         let json_value = serde_json::to_value(self)?;
 
         let params = match json_value {
@@ -1006,11 +1010,11 @@ impl Default for TextDocumentItemMetadata {
 }
 
 pub trait ToLSP<T> {
-    fn to_lsp(self) -> Result<T>;
+    fn to_lsp(&self) -> Result<T>;
 }
 
 impl ToLSP<Vec<FileEvent>> for notify::DebouncedEvent {
-    fn to_lsp(self) -> Result<Vec<FileEvent>> {
+    fn to_lsp(&self) -> Result<Vec<FileEvent>> {
         match self {
             notify::DebouncedEvent::Create(p) => Ok(vec![FileEvent {
                 uri: p.to_url()?,
diff --git a/src/utils.rs b/src/utils.rs
index 56f8a0507..f5caa2856 100644
--- a/src/utils.rs
+++ b/src/utils.rs
@@ -452,33 +452,6 @@ pub fn decode_parameter_label(
     }
 }
 
-/// Given a string, convert it into a string for vimscript
-/// The string gets surrounded by single quotes.
-///
-/// Existing single quotes will get escaped by inserting
-/// another single quote in place.
-///
-/// E.g.
-/// abcdefg -> 'abcdefg'
-/// abdcef'g -> 'abcdef''g'
-pub fn convert_to_vim_str(s: &str) -> String {
-    let mut vs = String::with_capacity(s.len());
-
-    vs.push('\'');
-
-    for i in s.chars() {
-        if i == '\'' {
-            vs.push(i);
-        }
-
-        vs.push(i);
-    }
-
-    vs.push('\'');
-
-    vs
-}
-
 /// Converts the kind of a `CodeAction` to a `&str`.
 pub fn code_action_kind_as_str(action: &CodeAction) -> &str {
     match action.kind.as_ref().map(|k| k.as_str()) {
@@ -766,13 +739,4 @@ mod test {
             }
         );
     }
-
-    #[test]
-    fn test_convert_to_vim_str() {
-        assert_eq!(convert_to_vim_str("abcdefg"), "'abcdefg'");
-        assert_eq!(convert_to_vim_str("'abcdefg"), "'''abcdefg'");
-        assert_eq!(convert_to_vim_str("'x'x'x'x'"), "'''x''x''x''x'''");
-        assert_eq!(convert_to_vim_str("xyz'''ffff"), "'xyz''''''ffff'");
-        assert_eq!(convert_to_vim_str("'''"), "''''''''");
-    }
 }
diff --git a/src/vim.rs b/src/vim.rs
index 1d0ed8841..215d7f0e2 100644
--- a/src/vim.rs
+++ b/src/vim.rs
@@ -29,7 +29,7 @@ pub struct HighlightSource {
     pub source: u64,
 }
 
-#[derive(Debug, Clone, Serialize, Deserialize)]
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
 pub struct Highlight {
     pub line: u32,
     pub character_start: u32,


Test patch: