Repository: microsoft/pyright
Pull request #10055: Implement support for pull diagnostics in Pyright
Author: login: rchiodo
Created: 2025-03-11T00:09:03Z | Merged: 2025-03-11T20:48:53Z

## Description

Currently whenever a file is opened or changed, Pyright analyzes the changed file (and its dependencies) on the BG thread and then sends a notification to the client with the list of diagnostics found. This is called 'push diagnostics'. https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_publishDiagnostics

This works fine as long as the associated file is actually open in the client. In some cases, the file may be open, but the user hasn't clicked on a tab or even the file may have been closed but we weren't told about it. 

See this issue for reference (which we already fixed in Pylance but not Pyright)
See https://github.com/microsoft/pylance-release/issues/5402

This causes issues with the problems list being out of sync with the actual state of the client. 

Additionally, Pyright has no notion of which files are more important than others to analyze (like maybe the one with the cursor in it).

This can be solved by using pull diagnostics instead:
https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_pullDiagnostics

Pull diagnostics lets the client decide what files it wants the list of diagnostics for. This allows it to prioritize say the actively editing file over other open tabs. It lets it remove diagnostics for files that are no longer open but maybe without having the LS server close the file (so that the next open will get diagnostics faster).

## Implementation

We already have an 'analyzeFile' and 'analyze' message for a per file diagnostics request, so implementation was just calling those methods. 

I also added two ways that pull will not be used. 

- diagnosticMode = 'workspace'. In this mode I left Pyright to analyze all of the files in the workspace itself and push notifications. 
- disablePullDiagnostics - this setting is a safety valve in case this breaks something. Users can enable this setting and we'll go back to the old method for diagnostics.

## Testing

I changed the languageServer.test.ts to have more tests that used pull diagnostics and tested myself with diagnosticMode on and off.




Diff:
diff --git a/packages/pyright-internal/src/analyzer/service.ts b/packages/pyright-internal/src/analyzer/service.ts
index f518369b75dc..5776c6efceac 100644
--- a/packages/pyright-internal/src/analyzer/service.ts
+++ b/packages/pyright-internal/src/analyzer/service.ts
@@ -91,6 +91,8 @@ export interface AnalyzerServiceOptions {
     serviceId?: string;
     skipScanningUserFiles?: boolean;
     fileSystem?: FileSystem;
+    usingPullDiagnostics?: boolean;
+    onInvalidated?: (reason: InvalidatedReason) => void;
 }
 
 interface ConfigFileContents {
@@ -229,6 +231,7 @@ export class AnalyzerService {
             backgroundAnalysis,
             skipScanningUserFiles: true,
             fileSystem,
+            usingPullDiagnostics: this.options.usingPullDiagnostics,
         });
 
         // Cloned service will use whatever user files the service currently has.
@@ -471,6 +474,10 @@ export class AnalyzerService {
     }
 
     invalidateAndForceReanalysis(reason: InvalidatedReason) {
+        if (this.options.onInvalidated) {
+            this.options.onInvalidated(reason);
+        }
+
         this._backgroundAnalysisProgram.invalidateAndForceReanalysis(reason);
     }
 
@@ -483,9 +490,13 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, the service doesn't perform analysis on its own.
+        // Instead the client deliberately asks for diagnostics on a file-by-file basis.
+        if (!this.options.usingPullDiagnostics) {
+            const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
+            if (moreToAnalyze) {
+                this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+            }
         }
     }
 
diff --git a/packages/pyright-internal/src/languageServerBase.ts b/packages/pyright-internal/src/languageServerBase.ts
index 29b5d95d36d3..a57365cc1652 100644
--- a/packages/pyright-internal/src/languageServerBase.ts
+++ b/packages/pyright-internal/src/languageServerBase.ts
@@ -33,6 +33,7 @@ import {
     Definition,
     DefinitionLink,
     Diagnostic,
+    DiagnosticRefreshRequest,
     DiagnosticRelatedInformation,
     DiagnosticSeverity,
     DiagnosticTag,
@@ -42,6 +43,8 @@ import {
     DidCloseTextDocumentParams,
     DidOpenTextDocumentParams,
     Disposable,
+    DocumentDiagnosticParams,
+    DocumentDiagnosticReport,
     DocumentHighlight,
     DocumentHighlightParams,
     DocumentSymbol,
@@ -50,6 +53,7 @@ import {
     HoverParams,
     InitializeParams,
     InitializeResult,
+    LSPObject,
     Location,
     MarkupKind,
     PrepareRenameParams,
@@ -57,18 +61,20 @@ import {
     ReferenceParams,
     RemoteWindow,
     RenameParams,
+    ResultProgressReporter,
     SignatureHelp,
     SignatureHelpParams,
     SymbolInformation,
     TextDocumentPositionParams,
     TextDocumentSyncKind,
     WorkDoneProgressReporter,
+    WorkspaceDiagnosticParams,
+    WorkspaceDocumentDiagnosticReport,
     WorkspaceEdit,
     WorkspaceFoldersChangeEvent,
     WorkspaceSymbol,
     WorkspaceSymbolParams,
 } from 'vscode-languageserver';
-import { ResultProgressReporter } from 'vscode-languageserver';
 
 import { TextDocument } from 'vscode-languageserver-textdocument';
 import { AnalysisResults } from './analyzer/analysis';
@@ -76,7 +82,7 @@ import { BackgroundAnalysisProgram, InvalidatedReason } from './analyzer/backgro
 import { ImportResolver } from './analyzer/importResolver';
 import { MaxAnalysisTime } from './analyzer/program';
 import { AnalyzerService, LibraryReanalysisTimeProvider, getNextServiceId } from './analyzer/service';
-import { IPythonMode } from './analyzer/sourceFile';
+import { IPythonMode, SourceFile } from './analyzer/sourceFile';
 import type { IBackgroundAnalysis } from './backgroundAnalysisBase';
 import { CommandResult } from './commands/commandResult';
 import { CancelAfter } from './common/cancellationUtils';
@@ -121,8 +127,10 @@ import { SignatureHelpProvider } from './languageService/signatureHelpProvider';
 import { WorkspaceSymbolProvider } from './languageService/workspaceSymbolProvider';
 import { Localizer, setLocaleOverride } from './localization/localize';
 import { ParseFileResults } from './parser/parser';
+import { ClientCapabilities, InitializationOptions } from './types';
 import { InitStatus, WellKnownWorkspaceKinds, Workspace, WorkspaceFactory } from './workspaceFactory';
-import { ClientCapabilities } from './types';
+
+const UncomputedDiagnosticsVersion = -1;
 
 export abstract class LanguageServerBase implements LanguageServerInterface, Disposable {
     // We support running only one "find all reference" at a time.
@@ -160,8 +168,8 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         supportsUnnecessaryDiagnosticTag: false,
         supportsTaskItemDiagnosticTag: false,
         completionItemResolveSupportsAdditionalTextEdits: false,
-        hasPullDiagnosticsCapability: false,
-        hasPullRelatedInformationCapability: false,
+        usingPullDiagnostics: false,
+        requiresPullRelatedInformationCapability: false,
     };
 
     protected defaultClientConfig: any;
@@ -269,6 +277,12 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             libraryReanalysisTimeProvider,
             serviceId,
             fileSystem: services?.fs ?? this.serverOptions.serviceProvider.fs(),
+            usingPullDiagnostics: this.client.usingPullDiagnostics,
+            onInvalidated: (reason) => {
+                if (this.client.usingPullDiagnostics) {
+                    this.connection.sendRequest(DiagnosticRefreshRequest.type);
+                }
+            },
         });
 
         service.setCompletionCallback((results) => this.onAnalysisCompletedHandler(service.fs, results));
@@ -484,6 +498,10 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.connection.onDidCloseTextDocument(async (params) => this.onDidCloseTextDocument(params));
         this.connection.onDidChangeWatchedFiles((params) => this.onDidChangeWatchedFiles(params));
 
+        this.connection.languages.diagnostics.on(async (params, token) => this.onDiagnostics(params, token));
+        this.connection.languages.diagnostics.onWorkspace(async (params, token) =>
+            this.onWorkspaceDiagnostics(params, token)
+        );
         this.connection.onExecuteCommand(async (params, token, reporter) =>
             this.onExecuteCommand(params, token, reporter)
         );
@@ -499,6 +517,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             setLocaleOverride(params.locale);
         }
 
+        const initializationOptions = (params.initializationOptions ?? {}) as LSPObject & InitializationOptions;
         const capabilities = params.capabilities;
         this.client.hasConfigurationCapability = !!capabilities.workspace?.configuration;
         this.client.hasWatchFileCapability = !!capabilities.workspace?.didChangeWatchedFiles?.dynamicRegistration;
@@ -543,6 +562,14 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             !!capabilities.textDocument?.completion?.completionItem?.resolveSupport?.properties.some(
                 (p) => p === 'additionalTextEdits'
             );
+        this.client.usingPullDiagnostics =
+            !!capabilities.textDocument?.diagnostic?.dynamicRegistration &&
+            initializationOptions?.diagnosticMode !== 'workspace' &&
+            initializationOptions?.disablePullDiagnostics !== true;
+        this.client.requiresPullRelatedInformationCapability =
+            !!capabilities.textDocument?.diagnostic?.relatedInformation &&
+            initializationOptions?.diagnosticMode !== 'workspace' &&
+            initializationOptions?.disablePullDiagnostics !== true;
 
         // Create a service instance for each of the workspace folders.
         this.workspaceFactory.handleInitialize(params);
@@ -602,6 +629,15 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             },
         };
 
+        if (this.client.usingPullDiagnostics) {
+            result.capabilities.diagnosticProvider = {
+                identifier: 'pyright',
+                documentSelector: null,
+                interFileDependencies: true,
+                workspaceDiagnostics: false, // Workspace wide are not pull diagnostics.
+            };
+        }
+
         return result;
     }
 
@@ -1061,6 +1097,88 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion - 1;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested && sourceFile) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                if (sourceFile) {
+                    await workspace.service.analyzeFile(uri, token);
+                }
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file. Diagnostics version should be reset to zero
+                // if a text edit comes in.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersionAfter.toString();
+            result.items = lspDiagnostics;
+        } else {
+            (result as any).kind = 'unchanged';
+            result.resultId = diagnosticsVersion.toString();
+            delete (result as any).items;
+        }
+
+        return result;
+    }
+
+    protected async onWorkspaceDiagnostics(params: WorkspaceDiagnosticParams, token: CancellationToken) {
+        const workspaces = await this.getWorkspaces();
+        const promises: Promise<WorkspaceDocumentDiagnosticReport>[] = [];
+        workspaces.forEach((workspace) => {
+            if (!workspace.disableLanguageServices) {
+                const files = workspace.service.getOwnedFiles();
+                files.forEach((file) => {
+                    const sourceFile = workspace.service.getSourceFile(file)!;
+                    if (canNavigateToFile(workspace.service.fs, sourceFile.getUri())) {
+                        promises.push(this._getWorkspaceDocumentDiagnostics(params, sourceFile, workspace, token));
+                    }
+                });
+            }
+        });
+        return {
+            items: await Promise.all(promises),
+        };
+    }
+
     protected onDidChangeWatchedFiles(params: DidChangeWatchedFilesParams) {
         params.changes.forEach((change) => {
             const filePath = this.fs.realCasePath(this.convertLspUriStringToUri(change.uri));
@@ -1151,6 +1269,11 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
     }
 
     protected onAnalysisCompletedHandler(fs: FileSystem, results: AnalysisResults): void {
+        // If we're in pull mode, disregard any 'tracking' results. They're not necessary.
+        if (this.client.usingPullDiagnostics && results.reason === 'tracking') {
+            return;
+        }
+
         // Send the computed diagnostics to the client.
         results.diagnostics.forEach((fileDiag) => {
             if (!this.canNavigateToFile(fileDiag.fileUri, fs)) {
@@ -1315,6 +1438,32 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
 
         return MarkupKind.PlainText;
     }
+    private async _getWorkspaceDocumentDiagnostics(
+        params: WorkspaceDiagnosticParams,
+        sourceFile: SourceFile,
+        workspace: Workspace,
+        token: CancellationToken
+    ) {
+        const originalUri = workspace.service.fs.getOriginalUri(sourceFile.getUri());
+        const result: WorkspaceDocumentDiagnosticReport = {
+            uri: originalUri.toString(),
+            version: sourceFile.getClientVersion() ?? null,
+            kind: 'full',
+            items: [],
+        };
+        const previousId = params.previousResultIds.find((x) => x.uri === originalUri.toString());
+        const documentResult = await this.onDiagnostics(
+            { previousResultId: previousId?.value, textDocument: { uri: result.uri } },
+            token
+        );
+        if (documentResult.kind === 'full') {
+            result.items = documentResult.items;
+        } else {
+            (result as any).kind = documentResult.kind;
+            delete (result as any).items;
+        }
+        return result;
+    }
 
     private _convertDiagnostics(fs: FileSystem, diags: AnalyzerDiagnostic[]): Diagnostic[] {
         const convertedDiags: Diagnostic[] = [];
diff --git a/packages/pyright-internal/src/tests/languageServer.test.ts b/packages/pyright-internal/src/tests/languageServer.test.ts
index c12f62ddca97..d67111644afe 100644
--- a/packages/pyright-internal/src/tests/languageServer.test.ts
+++ b/packages/pyright-internal/src/tests/languageServer.test.ts
@@ -41,7 +41,8 @@ describe(`Basic language server tests`, () => {
         callInitialize = true,
         extraSettings?: { item: ConfigurationItem; value: any }[],
         pythonVersion: PythonVersion = pythonVersion3_10,
-        supportsBackgroundThread?: boolean
+        supportsBackgroundThread?: boolean,
+        supportsPullDiagnostics?: boolean
     ) {
         const result = await runPyrightServer(
             projectRoots,
@@ -49,7 +50,8 @@ describe(`Basic language server tests`, () => {
             callInitialize,
             extraSettings,
             pythonVersion,
-            supportsBackgroundThread
+            supportsBackgroundThread,
+            supportsPullDiagnostics
         );
         serverInfo = result;
         return result;
@@ -71,7 +73,7 @@ describe(`Basic language server tests`, () => {
 // @filename: test.py
 //// # empty file
         `;
-        const serverInfo = await runLanguageServer(DEFAULT_WORKSPACE_ROOT, code, /* callInitialize */ false);
+        serverInfo = await runLanguageServer(DEFAULT_WORKSPACE_ROOT, code, /* callInitialize */ false);
 
         const initializeResult = await initializeLanguageServer(serverInfo);
 
@@ -149,90 +151,143 @@ describe(`Basic language server tests`, () => {
         assert(completionItem);
     });
 
-    test('background thread diagnostics', async () => {
-        const code = `
+    [false, true].forEach((supportsPullDiagnostics) => {
+        describe(`Diagnostics ${supportsPullDiagnostics ? 'pull' : 'push'}`, () => {
+            // Background analysis takes longer than 5 seconds sometimes, so we need to
+            // increase the timeout.
+            jest.setTimeout(15000);
+            test('background thread diagnostics', async () => {
+                const code = `
 // @filename: root/test.py
 //// from math import cos, sin
 //// import sys
 //// [|/*marker*/|]
         `;
-        const settings = [
-            {
-                item: {
-                    scopeUri: `file://${normalizeSlashes(DEFAULT_WORKSPACE_ROOT, '/')}`,
-                    section: 'python.analysis',
-                },
-                value: {
-                    typeCheckingMode: 'strict',
-                    diagnosticMode: 'workspace',
-                },
-            },
-        ];
+                const settings = [
+                    {
+                        item: {
+                            scopeUri: `file://${normalizeSlashes(DEFAULT_WORKSPACE_ROOT, '/')}`,
+                            section: 'python.analysis',
+                        },
+                        value: {
+                            typeCheckingMode: 'strict',
+                            diagnosticMode: 'workspace',
+                        },
+                    },
+                ];
 
-        const info = await runLanguageServer(
-            DEFAULT_WORKSPACE_ROOT,
-            code,
-            /* callInitialize */ true,
-            settings,
-            undefined,
-            /* supportsBackgroundThread */ true
-        );
+                const info = await runLanguageServer(
+                    DEFAULT_WORKSPACE_ROOT,
+                    code,
+                    /* callInitialize */ true,
+                    settings,
+                    undefined,
+                    /* supportsBackgroundThread */ true,
+                    supportsPullDiagnostics
+                );
 
-        // get the file containing the marker that also contains our task list comments
-        await openFile(info, 'marker');
+                // get the file containing the marker that also contains our task list comments
+                await openFile(info, 'marker');
 
-        // Wait for the diagnostics to publish
-        const diagnostics = await waitForDiagnostics(info);
-        const diagnostic = diagnostics.find((d) => d.uri.includes('root/test.py'));
-        assert(diagnostic);
-        assert.equal(diagnostic.diagnostics.length, 6);
+                // Wait for the diagnostics to publish
+                const diagnostics = await waitForDiagnostics(info);
+                const diagnostic = diagnostics.find((d) => d.uri.includes('root/test.py'));
+                assert(diagnostic);
+                assert.equal(diagnostic.diagnostics.length, 6);
 
-        // Make sure the error has a special rule
-        assert.equal(diagnostic.diagnostics[1].code, 'reportUnusedImport');
-        assert.equal(diagnostic.diagnostics[3].code, 'reportUnusedImport');
-        assert.equal(diagnostic.diagnostics[5].code, 'reportUnusedImport');
-    });
+                // Make sure the error has a special rule
+                assert.equal(diagnostic.diagnostics[1].code, 'reportUnusedImport');
+                assert.equal(diagnostic.diagnostics[3].code, 'reportUnusedImport');
+                assert.equal(diagnostic.diagnostics[5].code, 'reportUnusedImport');
+            });
 
-    test('Diagnostic severity overrides test', async () => {
-        const code = `
+            test('background thread diagnostics open mode', async () => {
+                const code = `
+// @filename: root/test.py
+//// from math import cos, sin
+//// import sys
+//// [|/*marker*/|]
+        `;
+                const settings = [
+                    {
+                        item: {
+                            scopeUri: `file://${normalizeSlashes(DEFAULT_WORKSPACE_ROOT, '/')}`,
+                            section: 'python.analysis',
+                        },
+                        value: {
+                            typeCheckingMode: 'strict',
+                        },
+                    },
+                ];
+
+                const info = await runLanguageServer(
+                    DEFAULT_WORKSPACE_ROOT,
+                    code,
+                    /* callInitialize */ true,
+                    settings,
+                    undefined,
+                    /* supportsBackgroundThread */ true,
+                    supportsPullDiagnostics
+                );
+
+                // get the file containing the marker that also contains our task list comments
+                await openFile(info, 'marker');
+
+                // Wait for the diagnostics to publish
+                const diagnostics = await waitForDiagnostics(info);
+                const diagnostic = diagnostics.find((d) => d.uri.includes('root/test.py'));
+                assert(diagnostic);
+                assert.equal(diagnostic.diagnostics.length, 6);
+
+                // Make sure the error has a special rule
+                assert.equal(diagnostic.diagnostics[1].code, 'reportUnusedImport');
+                assert.equal(diagnostic.diagnostics[3].code, 'reportUnusedImport');
+                assert.equal(diagnostic.diagnostics[5].code, 'reportUnusedImport');
+            });
+
+            test('Diagnostic severity overrides test', async () => {
+                const code = `
 // @filename: test.py
 //// def test([|/*marker*/x|]): ...
 //// 
 // @filename: pyproject.toml
 //// 
     `;
-        const settings = [
-            {
-                item: {
-                    scopeUri: `file://${normalizeSlashes(DEFAULT_WORKSPACE_ROOT, '/')}`,
-                    section: 'python.analysis',
-                },
-                value: {
-                    diagnosticSeverityOverrides: {
-                        reportUnknownParameterType: 'warning',
+                const settings = [
+                    {
+                        item: {
+                            scopeUri: `file://${normalizeSlashes(DEFAULT_WORKSPACE_ROOT, '/')}`,
+                            section: 'python.analysis',
+                        },
+                        value: {
+                            diagnosticSeverityOverrides: {
+                                reportUnknownParameterType: 'warning',
+                            },
+                        },
                     },
-                },
-            },
-        ];
+                ];
 
-        const info = await runLanguageServer(
-            DEFAULT_WORKSPACE_ROOT,
-            code,
-            /* callInitialize */ true,
-            settings,
-            undefined,
-            /* supportsBackgroundThread */ true
-        );
+                const info = await runLanguageServer(
+                    DEFAULT_WORKSPACE_ROOT,
+                    code,
+                    /* callInitialize */ true,
+                    settings,
+                    undefined,
+                    /* supportsBackgroundThread */ true,
+                    supportsPullDiagnostics
+                );
 
-        // get the file containing the marker that also contains our task list comments
-        await openFile(info, 'marker');
+                // get the file containing the marker that also contains our task list comments
+                await openFile(info, 'marker');
 
-        // Wait for the diagnostics to publish
-        const diagnostics = await waitForDiagnostics(info);
-        const diagnostic = diagnostics.find((d) => d.uri.includes('test.py'));
-        assert(diagnostic);
+                // Wait for the diagnostics to publish
+                const diagnostics = await waitForDiagnostics(info);
+                const diagnostic = diagnostics.find((d) => d.uri.includes('test.py'));
+                assert(diagnostic);
 
-        // Make sure the error has a special rule
-        assert.equal(diagnostic.diagnostics[0].code, 'reportUnknownParameterType');
+                // Make sure the error has a special rule
+                assert.equal(diagnostic.diagnostics[0].code, 'reportUnknownParameterType');
+            });
+        });
     });
 });
diff --git a/packages/pyright-internal/src/tests/lsp/customLsp.ts b/packages/pyright-internal/src/tests/lsp/customLsp.ts
index 5a6d8379a5c7..a1158bdb528b 100644
--- a/packages/pyright-internal/src/tests/lsp/customLsp.ts
+++ b/packages/pyright-internal/src/tests/lsp/customLsp.ts
@@ -55,6 +55,7 @@ export namespace CustomLSP {
 
     export enum Requests {
         GetDiagnostics = 'test/getDiagnostics',
+        GetOpenFiles = 'test/getOpenFiles',
     }
 
     export enum Notifications {
@@ -86,6 +87,7 @@ export namespace CustomLSP {
 
     interface Params {
         [Requests.GetDiagnostics]: { uri: string };
+        [Requests.GetOpenFiles]: { uri: string };
         [Notifications.CacheDirCreate]: { uri: string };
         [Notifications.CacheFileWrite]: { uri: string; contents: string; overwrite: boolean };
         [Notifications.SetStatusBarMessage]: string;
@@ -102,6 +104,7 @@ export namespace CustomLSP {
 
     interface Response {
         [Requests.GetDiagnostics]: { diagnostics: string };
+        [Requests.GetOpenFiles]: { files: string };
     }
 
     // Interface for returning config options as we cannot return a
diff --git a/packages/pyright-internal/src/tests/lsp/languageServer.ts b/packages/pyright-internal/src/tests/lsp/languageServer.ts
index a468935e8f16..145ebbf5de0e 100644
--- a/packages/pyright-internal/src/tests/lsp/languageServer.ts
+++ b/packages/pyright-internal/src/tests/lsp/languageServer.ts
@@ -203,6 +203,11 @@ async function runServer(
                 const file = workspace.service.test_program.getBoundSourceFile(filePath);
                 const diagnostics = file?.getDiagnostics(workspace.service.test_program.configOptions) || [];
                 return { diagnostics: serialize(diagnostics) };
+            }),
+            CustomLSP.onRequest(connection, CustomLSP.Requests.GetOpenFiles, async (params) => {
+                const workspace = await server.getWorkspaceForFile(Uri.parse(params.uri, server.serviceProvider));
+                const files = serialize(workspace.service.test_program.getOpened().map((f) => f.sourceFile.getUri()));
+                return { files: files };
             })
         );
 
@@ -403,3 +408,13 @@ export async function run() {
         stateManager.run();
     }
 }
+
+process.on('uncaughtException', (err) => {
+    console.error('Uncaught exception in worker:', err);
+    process.exit(10); // Exit the worker process
+});
+
+process.on('unhandledRejection', (reason, promise) => {
+    console.error('Unhandled rejection in worker:', reason);
+    process.exit(11); // Exit the worker process
+});
diff --git a/packages/pyright-internal/src/tests/lsp/languageServerTestUtils.ts b/packages/pyright-internal/src/tests/lsp/languageServerTestUtils.ts
index f29c82b25683..dd10350e2f80 100644
--- a/packages/pyright-internal/src/tests/lsp/languageServerTestUtils.ts
+++ b/packages/pyright-internal/src/tests/lsp/languageServerTestUtils.ts
@@ -16,9 +16,13 @@ import {
     CancellationToken,
     ConfigurationItem,
     ConfigurationRequest,
+    DiagnosticRefreshRequest,
     DidChangeWorkspaceFoldersNotification,
     DidOpenTextDocumentNotification,
     Disposable,
+    DocumentDiagnosticReport,
+    DocumentDiagnosticRequest,
+    FullDocumentDiagnosticReport,
     InitializedNotification,
     InitializeParams,
     InitializeRequest,
@@ -32,7 +36,10 @@ import {
     SemanticTokensRefreshRequest,
     ShutdownRequest,
     TelemetryEventNotification,
+    UnchangedDocumentDiagnosticReport,
     UnregistrationRequest,
+    WorkspaceDiagnosticReport,
+    WorkspaceDiagnosticRequest,
 } from 'vscode-languageserver-protocol';
 import {
     Connection,
@@ -52,6 +59,7 @@ import {
 } from 'vscode-languageserver/node';
 import { PythonPathResult } from '../../analyzer/pythonPathUtils';
 import { IPythonMode } from '../../analyzer/sourceFile';
+import { deserialize } from '../../backgroundThreadBase';
 import { PythonPlatform } from '../../common/configOptions';
 import { toBoolean } from '../../common/core';
 import { createDeferred, Deferred } from '../../common/deferred';
@@ -100,6 +108,7 @@ export interface PyrightServerInfo {
     progressReporterStatus: Map<string, number>;
     progressParts: Map<string, TestProgressPart>;
     telemetry: any[];
+    supportsPullDiagnostics: boolean;
     diagnostics: PublishDiagnosticsParams[];
     diagnosticsEvent: Event<PublishDiagnosticsParams>;
     workspaceEdits: ApplyWorkspaceEditParams[];
@@ -188,6 +197,7 @@ function createServerWorker(file: string, testServerData: CustomLSP.TestServerSt
     });
     serverWorker.on('exit', (code) => {
         logToDisk(`Worker exit: ${code}`, testServerData.logFile);
+        serverWorker = undefined;
     });
     return serverWorker;
 }
@@ -268,7 +278,17 @@ function createServerConnection(testServerData: CustomLSP.TestServerStartOptions
     return connection;
 }
 
-export async function waitForDiagnostics(info: PyrightServerInfo, timeout = 10000) {
+function getProjectRootString(info: PyrightServerInfo, projectRoot?: Uri) {
+    return projectRoot ? projectRoot.toString() : info.projectRoots.length > 0 ? info.projectRoots[0].toString() : '';
+}
+
+export async function getOpenFiles(info: PyrightServerInfo, projectRoot?: Uri): Promise<Uri[]> {
+    const uri = getProjectRootString(info, projectRoot);
+    const result = await CustomLSP.sendRequest(info.connection, CustomLSP.Requests.GetOpenFiles, { uri });
+    return deserialize(result.files);
+}
+
+async function waitForPushDiagnostics(info: PyrightServerInfo, timeout = 10000) {
     const deferred = createDeferred<void>();
     const disposable = info.diagnosticsEvent((params) => {
         if (params.diagnostics.length > 0) {
@@ -285,6 +305,82 @@ export async function waitForDiagnostics(info: PyrightServerInfo, timeout = 1000
     return info.diagnostics;
 }
 
+export async function waitForEvent<T>(event: Event<T>, name: string, condition: (p: T) => boolean, timeout = 10000) {
+    const deferred = createDeferred<void>();
+    const disposable = event((params) => {
+        if (condition(params)) {
+            deferred.resolve();
+        }
+    });
+
+    const timer = setTimeout(() => deferred.reject(`Timed out waiting for ${name} event`), timeout);
+
+    try {
+        await deferred.promise;
+    } finally {
+        clearTimeout(timer);
+        disposable.dispose();
+    }
+}
+
+function convertDiagnosticReportItem(
+    uri: string,
+    item: FullDocumentDiagnosticReport | UnchangedDocumentDiagnosticReport
+): PublishDiagnosticsParams {
+    if (item.kind === 'unchanged') {
+        return {
+            uri,
+            diagnostics: [],
+        };
+    }
+
+    return {
+        uri,
+        diagnostics: item.items,
+    };
+}
+export function convertDiagnosticReport(
+    uri: string | undefined,
+    report: DocumentDiagnosticReport | WorkspaceDiagnosticReport
+): PublishDiagnosticsParams[] {
+    if (!(report as any).kind || !uri) {
+        const workspaceReport = report as WorkspaceDiagnosticReport;
+        return workspaceReport.items.map((item) => convertDiagnosticReportItem(item.uri, item));
+    }
+    const documentReport = report as DocumentDiagnosticReport;
+    return [convertDiagnosticReportItem(uri, documentReport)];
+}
+
+async function waitForPullDiagnostics(info: PyrightServerInfo): Promise<PublishDiagnosticsParams[]> {
+    const openFiles = await getOpenFiles(info);
+    if (openFiles.length <= 0) {
+        const results = await info.connection.sendRequest(WorkspaceDiagnosticRequest.type, {
+            identifier: 'Pylance',
+            previousResultIds: [],
+        });
+        return convertDiagnosticReport(undefined, results);
+    } else {
+        const results: PublishDiagnosticsParams[] = [];
+        for (const openFile of openFiles) {
+            const result = await info.connection.sendRequest(DocumentDiagnosticRequest.type, {
+                textDocument: {
+                    uri: openFile.toString(),
+                },
+            });
+            results.push(convertDiagnosticReport(openFile.toString(), result)[0]);
+        }
+        return results;
+    }
+}
+
+export async function waitForDiagnostics(info: PyrightServerInfo, timeout = 20000) {
+    if (info.supportsPullDiagnostics) {
+        // Timeout doesn't apply on pull because we can actually ask for them.
+        return waitForPullDiagnostics(info);
+    }
+    return waitForPushDiagnostics(info, timeout);
+}
+
 interface ProgressPart {}
 
 interface ProgressContext {
@@ -329,7 +425,8 @@ export async function runPyrightServer(
     callInitialize = true,
     extraSettings?: { item: ConfigurationItem; value: any }[],
     pythonVersion: PythonVersion = pythonVersion3_10,
-    backgroundAnalysis?: boolean
+    backgroundAnalysis?: boolean,
+    supportPullDiagnostics?: boolean
 ): Promise<PyrightServerInfo> {
     // Setup the test data we need to send for Test server startup.
     const projectRootsArray = Array.isArray(projectRoots) ? projectRoots : [projectRoots];
@@ -372,6 +469,7 @@ export async function runPyrightServer(
     const serverStarted = createDeferred<string>();
     const diagnosticsEmitter = new Emitter<PublishDiagnosticsParams>();
     const workspaceEditsEmitter = new Emitter<ApplyWorkspaceEditParams>();
+    const diagnosticsMode = extraSettings?.find((s) => s.item.section === 'python.analysis')?.value?.diagnosticMode;
 
     // Setup the server info.
     const info: PyrightServerInfo = {
@@ -386,16 +484,20 @@ export async function runPyrightServer(
         testData,
         testName: testServerData.testName,
         telemetry: [],
+        supportsPullDiagnostics: (supportPullDiagnostics && diagnosticsMode !== 'workspace') ?? false,
         projectRoots: testServerData.projectRoots,
         diagnostics: [],
         diagnosticsEvent: diagnosticsEmitter.event,
         workspaceEdits: [],
         workspaceEditsEvent: workspaceEditsEmitter.event,
-        getInitializeParams: () => getInitializeParams(testServerData.projectRoots),
+        getInitializeParams: () =>
+            getInitializeParams(testServerData.projectRoots, !!supportPullDiagnostics, diagnosticsMode),
         convertPathToUri: (path: string) => UriEx.file(path, !ignoreCase),
         dispose: async () => {
             // Send shutdown. This should disconnect the dispatcher and kill the server.
-            await connection.sendRequest(ShutdownRequest.type, undefined);
+            if (serverWorker) {
+                await connection.sendRequest(ShutdownRequest.type, undefined);
+            }
 
             // Now we can dispose the connection.
             disposables.forEach((d) => d.dispose());
@@ -422,6 +524,7 @@ export async function runPyrightServer(
         info.connection.onRequest(InlayHintRefreshRequest.type, () => {
             // Empty. Silently ignore for now.
         }),
+        info.connection.onRequest(DiagnosticRefreshRequest.type, () => {}),
         info.connection.onRequest(ApplyWorkspaceEditRequest.type, (p) => {
             info.workspaceEdits.push(p);
             workspaceEditsEmitter.fire(p);
@@ -574,7 +677,11 @@ export async function hover(info: PyrightServerInfo, markerName: string) {
     return hoverResult;
 }
 
-export function getInitializeParams(projectRoots: Uri[]) {
+export function getInitializeParams(
+    projectRoots: Uri[],
+    supportsPullDiagnostics: boolean,
+    diagnosticMode: string | undefined = undefined
+) {
     // cloned vscode "1.71.0-insider"'s initialize params.
     const workspaceFolders = projectRoots
         ? projectRoots.map((root, i) => ({ name: root.fileName, uri: projectRoots[i].toString() }))
@@ -927,6 +1034,8 @@ export function getInitializeParams(projectRoots: Uri[]) {
         },
         initializationOptions: {
             autoFormatStrings: true,
+            diagnosticMode: diagnosticMode ?? 'openFilesOnly',
+            disablePullDiagnostics: !supportsPullDiagnostics,
         },
         workspaceFolders,
     };
diff --git a/packages/pyright-internal/src/types.ts b/packages/pyright-internal/src/types.ts
index 31ff83823ddf..e06c99e22d80 100644
--- a/packages/pyright-internal/src/types.ts
+++ b/packages/pyright-internal/src/types.ts
@@ -29,6 +29,11 @@ export interface ClientCapabilities {
     supportsUnnecessaryDiagnosticTag: boolean;
     supportsTaskItemDiagnosticTag: boolean;
     completionItemResolveSupportsAdditionalTextEdits: boolean;
-    hasPullDiagnosticsCapability: boolean;
-    hasPullRelatedInformationCapability: boolean;
+    usingPullDiagnostics: boolean;
+    requiresPullRelatedInformationCapability: boolean;
 }
+
+export type InitializationOptions = {
+    diagnosticMode?: string;
+    disablePullDiagnostics?: boolean;
+};
diff --git a/packages/vscode-pyright/package.json b/packages/vscode-pyright/package.json
index c51797453544..c572ca1f2286 100644
--- a/packages/vscode-pyright/package.json
+++ b/packages/vscode-pyright/package.json
@@ -1545,6 +1545,12 @@
                     "description": "Disables the “Organize Imports” command.",
                     "scope": "resource"
                 },
+                "pyright.disablePullDiagnostics": {
+                    "type": "boolean",
+                    "default": false,
+                    "description": "Disables the use of pull diagnostics from VS Code.",
+                    "scope": "machine"
+                },
                 "python.pythonPath": {
                     "type": "string",
                     "default": "python",
diff --git a/packages/vscode-pyright/src/extension.ts b/packages/vscode-pyright/src/extension.ts
index 2ae8228353f1..70945457a132 100644
--- a/packages/vscode-pyright/src/extension.ts
+++ b/packages/vscode-pyright/src/extension.ts
@@ -175,6 +175,10 @@ export async function activate(context: ExtensionContext) {
                 },
             },
         },
+        initializationOptions: {
+            diagnosticMode: workspace.getConfiguration('python.analysis').get('diagnosticMode'),
+            disablePullDiagnostics: workspace.getConfiguration('pyright').get('disablePullDiagnostics'),
+        },
     };
 
     // Create the language client and start the client.


Reviews:
user: login: erictraut
body: LGTM. It might be worth doing a quick manual test of the CLI with watch mode (--watch). We don't have any tests for this, and it could potentially be affected by some of the code paths you touched in this PR
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-11T16:33:09Z
user: login: heejaechang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:45:44Z
user: login: heejaechang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:47:11Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:47:15Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:48:02Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:50:29Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:55:40Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:17:27Z
user: login: debonte
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T19:19:09Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:21:40Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:24:38Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:25:25Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:30:36Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:31:31Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:33:23Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T19:33:38Z
user: login: debonte
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T19:36:26Z
user: login: debonte
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T19:38:40Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T20:07:56Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T20:21:02Z
user: login: rchiodo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T20:35:23Z

Review comments:
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: heejaechang
body: won't this cause all open files to get pull diagnostics request for every char changes?
created_at: 2025-03-11T18:45:43Z
author_association: COLLABORATOR
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: heejaechang
body: pylance had similar stuff for semantic token and inlay hint and @bschnurr removed those due to perf issue?
created_at: 2025-03-11T18:47:11Z
author_association: COLLABORATOR
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: rchiodo
body: You think this would be better inside of the timer during scheduleReanalysis?

created_at: 2025-03-11T18:47:15Z
author_association: COLLABORATOR
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: rchiodo
body: I believe VS code is smart enough to only queue up 1 request after a change happens, but let me double check
created_at: 2025-03-11T18:48:02Z
author_association: COLLABORATOR
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: rchiodo
body: No, you're right. It causes double requests. We don't actually compute anything on the second request (as the previous result id matches) but it's still sending them.
created_at: 2025-03-11T18:50:29Z
author_association: COLLABORATOR
diff_hunk: @@ -602,6 +628,15 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             },
         };
 
+        if (this.client.hasPullDiagnosticsCapability) {
+            result.capabilities.diagnosticProvider = {
+                identifier: 'pyright',
+                documentSelector: null,
+                interFileDependencies: true,
+                workspaceDiagnostics: initializationOptions.diagnosticMode === 'workspace',
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: Isn't it impossible for `diagnosticMode` to be set to `workspace` here, since that would have resulted in `hasPullDiagnosticsCapability` being forced to `false`?
created_at: 2025-03-11T18:51:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -543,6 +561,14 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             !!capabilities.textDocument?.completion?.completionItem?.resolveSupport?.properties.some(
                 (p) => p === 'additionalTextEdits'
             );
+        this.client.hasPullDiagnosticsCapability =
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: I find these two new `hasFooCapability` names confusing since they're a lie. They could be `false` even though the client has the capability. Maybe `canUsePullDiagnostics` or something like that?
created_at: 2025-03-11T18:52:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: rchiodo
body: Oh actually this is already called during the timer
created_at: 2025-03-11T18:55:40Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: Do we intentionally call `analyzeFile` when `sourceFile` is `undefined`?
created_at: 2025-03-11T18:59:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: I'm confused about the `SourceFile` objects in play within this region of the code. We check if `sourceFile` is defined before the loop, but then within the loop we re-get it. And then re-get it at the bottom of the loop -- why would the `SourceFile` be a different object after analysis?
created_at: 2025-03-11T19:03:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: It's not clear to me how we're doing what this comment says we're doing. Wouldn't we need to check the file content version (or whatever it's called) to make this happen?
created_at: 2025-03-11T19:11:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: Should this be `diagnosticsVersionAfter`?
created_at: 2025-03-11T19:11:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
+            result.items = lspDiagnostics;
+        } else {
+            (result as any).kind = 'unchanged';
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: If `sourceFile` is `undefined` I believe will set `resultId` to `"-1"` here. Is that ok? It seemed to me that the logic above assumes that `-1` is a secret value that we only use internally and will never be sent by LSP as the `previousResultId`.
created_at: 2025-03-11T19:14:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -483,9 +500,16 @@ export class AnalyzerService {
     }
 
     protected runAnalysis(token: CancellationToken) {
-        const moreToAnalyze = this._backgroundAnalysisProgram.startAnalysis(token);
-        if (moreToAnalyze) {
-            this._scheduleReanalysis(/* requireTrackedFileUpdate */ false);
+        // In pull diagnostics mode, ask VS code to refresh the diagnostics. This is how
+        // we tell VS code that something happened that might affect diagnostics.
+        if (this.options.usingPullDiagnostics) {
+            this._connection?.sendRequest(DiagnosticRefreshRequest.type);
path: packages/pyright-internal/src/analyzer/service.ts
user: login: rchiodo
body: I changed it to be a callback and only happen during 'invalidateAndForceReanalysis'. This should handle the cases that VS code doesn't already cause an update for
created_at: 2025-03-11T19:17:27Z
author_association: COLLABORATOR
diff_hunk: @@ -602,6 +628,15 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             },
         };
 
+        if (this.client.hasPullDiagnosticsCapability) {
+            result.capabilities.diagnosticProvider = {
+                identifier: 'pyright',
+                documentSelector: null,
+                interFileDependencies: true,
+                workspaceDiagnostics: initializationOptions.diagnosticMode === 'workspace',
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: Yeah that's true. This will always be false.
created_at: 2025-03-11T19:21:40Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: No it's not intentional. sourceFile should never be null here, but I'll add a check.
created_at: 2025-03-11T19:24:38Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: The file could theoretically be closed/deleted while analysis was happening
created_at: 2025-03-11T19:25:25Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: Yes, it's checking the diagnostics version. That's how it checks. Diagnostics version would be zero if an update occurred so it should go back around.

This code here causes that (inside the program.setFileOpened call)

```
            // Reset the diagnostic version so we force an update to the
            // diagnostics, which can change based on whether the file is open.
            // We do not set the version to undefined here because that implies
            // there are no diagnostics currently reported for this file.
            sourceFileInfo.diagnosticsVersion = 0;
```
created_at: 2025-03-11T19:30:36Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: I don't think so. They're the same as long as the loop exits.
created_at: 2025-03-11T19:31:31Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
+            result.items = lspDiagnostics;
+        } else {
+            (result as any).kind = 'unchanged';
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: LSP says it's a string, so it can be anything. Next time through it just passes it to us, so I think -1 is fine here.
created_at: 2025-03-11T19:33:23Z
author_association: COLLABORATOR
diff_hunk: @@ -543,6 +561,14 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
             !!capabilities.textDocument?.completion?.completionItem?.resolveSupport?.properties.some(
                 (p) => p === 'additionalTextEdits'
             );
+        this.client.hasPullDiagnosticsCapability =
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: I'll rename them.
created_at: 2025-03-11T19:33:38Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: Ah, right, of course. For what it's worth, I found this loop extremely confusing. I stared at it for a long time trying to figure out what was going on here. I don't have specific recommendations on how to improve it unfortunately. There was some similar code that I reviewed in pyrx recently that I also found very difficult to wrap my head around. Might just be me.
created_at: 2025-03-11T19:36:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
+            result.items = lspDiagnostics;
+        } else {
+            (result as any).kind = 'unchanged';
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: debonte
body: Right, but if `previousResultId` is `-1` and the file needs checking we'll set `diagnosticsVersion` to `-1` and then the analysis loop will get skipped by the `if (params.previousResultId !== diagnosticsVersion.toString()` check. Won't it?

My assumption is that the `resultId` we return here is going to be sent back to us as `previousResultId` in the next pull request.
created_at: 2025-03-11T19:38:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: Oh I guess if sourceFile is suddenly undefined, the after one would be more accurate.
created_at: 2025-03-11T20:07:56Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: Diagnostics version also updates when things are invalidated because of say an imported file changing
created_at: 2025-03-11T20:21:02Z
author_association: COLLABORATOR
diff_hunk: @@ -1061,6 +1096,85 @@ export abstract class LanguageServerBase implements LanguageServerInterface, Dis
         this.openFileMap.delete(uri.key);
     }
 
+    protected async onDiagnostics(params: DocumentDiagnosticParams, token: CancellationToken) {
+        const uri = this.convertLspUriStringToUri(params.textDocument.uri);
+        const workspace = await this.getWorkspaceForFile(uri);
+        let sourceFile = workspace.service.getSourceFile(uri);
+        let diagnosticsVersion = sourceFile?.isCheckingRequired()
+            ? UncomputedDiagnosticsVersion
+            : sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+        const result: DocumentDiagnosticReport = {
+            kind: 'full',
+            resultId: sourceFile?.getDiagnosticVersion()?.toString(),
+            items: [],
+        };
+        if (
+            workspace.disableLanguageServices ||
+            !canNavigateToFile(workspace.service.fs, uri) ||
+            token.isCancellationRequested
+        ) {
+            return result;
+        }
+
+        // Reanalyze the file if it's not up to date.
+        if (params.previousResultId !== diagnosticsVersion.toString() && sourceFile) {
+            let diagnosticsVersionAfter = UncomputedDiagnosticsVersion - 1; // Just has to be different
+            let serverDiagnostics: AnalyzerDiagnostic[] = [];
+
+            // Loop until we analyze the same version that we started with.
+            while (diagnosticsVersion !== diagnosticsVersionAfter && !token.isCancellationRequested) {
+                // Reset the version we're analyzing
+                sourceFile = workspace.service.getSourceFile(uri);
+                diagnosticsVersion = sourceFile?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+
+                // Then reanalyze the file (this should go to the background thread so this thread can handle other requests).
+                await workspace.service.analyzeFile(uri, token);
+
+                // Then pick up the diagnostics created.
+                serverDiagnostics = sourceFile
+                    ? await workspace.service.getDiagnosticsForRange(uri, sourceFile.getRange(), token)
+                    : [];
+
+                // If any text edits came in, make sure we reanalyze the file.
+                const sourceFileAfter = workspace.service.getSourceFile(uri);
+                diagnosticsVersionAfter = sourceFileAfter?.getDiagnosticVersion() ?? UncomputedDiagnosticsVersion;
+            }
+
+            // Then convert the diagnostics to the LSP format.
+            const lspDiagnostics = this._convertDiagnostics(workspace.service.fs, serverDiagnostics).filter(
+                (d) => d !== undefined
+            ) as Diagnostic[];
+
+            result.resultId = diagnosticsVersion.toString();
+            result.items = lspDiagnostics;
+        } else {
+            (result as any).kind = 'unchanged';
+            result.resultId = diagnosticsVersion.toString();
path: packages/pyright-internal/src/languageServerBase.ts
user: login: rchiodo
body: I forgot about the isCheckingRequired test. I think you're right. It should be -2.
created_at: 2025-03-11T20:35:23Z
author_association: COLLABORATOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-11T00:17:44Z
body: Diff from [mypy_primer](https://github.com/hauntsaninja/mypy_primer), showing the effect of this PR on open source code:
```diff
sympy (https://github.com/sympy/sympy)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:120:18 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:120:18 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:123:18 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:123:18 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:126:9 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:126:9 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:132:9 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:132:9 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:895:12 - error: Operator "/" not supported for types "Equality | Unknown | Relational | Unequality | Integer" and "Literal[2]"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:895:12 - error: Operator "/" not supported for types "Equality | Unknown" and "Literal[2]"
-     Operator "/" not supported for types "Equality" and "Literal[2]"
-     Operator "/" not supported for types "Relational" and "Literal[2]"
-     Operator "/" not supported for types "Unequality" and "Literal[2]" (reportOperatorIssue)
+     Operator "/" not supported for types "Equality" and "Literal[2]" (reportOperatorIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1863:12 - error: Cannot access attribute "combsimp" for class "Relational"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1863:12 - error: Cannot access attribute "combsimp" for class "Unequality"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1911:14 - error: Cannot access attribute "factor" for class "Relational"
-     Attribute "factor" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1911:14 - error: Cannot access attribute "factor" for class "Unequality"
-     Attribute "factor" is unknown (reportAttributeAccessIssue)
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1918:14 - error: Cannot access attribute "combsimp" for class "Relational"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1918:14 - error: Cannot access attribute "combsimp" for class "Unequality"
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1932:12 - error: Cannot access attribute "combsimp" for class "Relational"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1932:12 - error: Cannot access attribute "combsimp" for class "Unequality"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
-     Type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" is not assignable to type "ConvertibleToFloat"
+     Type "Equality | Unknown | NaN | Sum" is not assignable to type "ConvertibleToFloat"
-       Type "Relational" is not assignable to type "ConvertibleToFloat"
+       Type "Equality" is not assignable to type "ConvertibleToFloat"
-         "Relational" is not assignable to "str"
+         "Equality" is not assignable to "str"
-         "Relational" is incompatible with protocol "Buffer"
+         "Equality" is incompatible with protocol "Buffer"
-         "Relational" is incompatible with protocol "SupportsFloat"
+         "Equality" is incompatible with protocol "SupportsFloat"
-         "Relational" is incompatible with protocol "SupportsIndex"
+         "Equality" is incompatible with protocol "SupportsIndex"
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
-     Type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" is not assignable to type "ConvertibleToFloat"
+     Type "Equality | Unknown | NaN | Sum" is not assignable to type "ConvertibleToFloat"
-       Type "Relational" is not assignable to type "ConvertibleToFloat"
+       Type "Equality" is not assignable to type "ConvertibleToFloat"
-         "Relational" is not assignable to "str"
+         "Equality" is not assignable to "str"
-         "Relational" is incompatible with protocol "Buffer"
+         "Equality" is incompatible with protocol "Buffer"
-         "Relational" is incompatible with protocol "SupportsFloat"
+         "Equality" is incompatible with protocol "SupportsFloat"
-         "Relational" is incompatible with protocol "SupportsIndex" (reportArgumentType)
+         "Equality" is incompatible with protocol "SupportsIndex" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2074:56 - error: Cannot access attribute "gammasimp" for class "Relational"
-     Attribute "gammasimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2074:56 - error: Cannot access attribute "gammasimp" for class "Unequality"
-     Attribute "gammasimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2973:37 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2973:37 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | NaN | Sum"
-     Operator "*" not supported for types "Literal[2]" and "Equality"
-     Operator "*" not supported for types "Literal[2]" and "Relational"
-     Operator "*" not supported for types "Literal[2]" and "Unequality" (reportOperatorIssue)
+     Operator "*" not supported for types "Literal[2]" and "Equality" (reportOperatorIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2978:17 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2978:17 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | NaN | Sum"
-     Operator "*" not supported for types "Literal[2]" and "Equality"
-     Operator "*" not supported for types "Literal[2]" and "Relational"
-     Operator "*" not supported for types "Literal[2]" and "Unequality" (reportOperatorIssue)

... (truncated 66 lines) ...

```
author_association: CONTRIBUTOR
user: login: erictraut
created_at: 2025-03-11T07:17:46Z
body: Is there any chance this will break the language server when it's running with clients other than VS Code? Does it fall back on push diagnostics if the client doesn't support the pull mechanism?
author_association: COLLABORATOR
user: login: rchiodo
created_at: 2025-03-11T15:58:48Z
body: >Is there any chance this will break the language server when it's running with clients other than VS Code? 

Well there's always that chance but it's implementing the LSP spec for pull diagnostics. As long as those other editors make the same pull calls, it should work. There's nothing VS code specific about the handling of the pull requests.

> Does it fall back on push diagnostics if the client doesn't support the pull mechanism?

Yes, the client has to have these settings in the initialization params:

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#diagnosticClientCapabilities

If those aren't set, push diagnostics will still be used. We have to return this setting back in the result to indicate, yes we support pull diagnostics:

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#diagnosticOptions

Do you have any ideas on which IDEs would be a good test bed? I should try that to make sure.
author_association: COLLABORATOR
user: login: erictraut
created_at: 2025-03-11T16:28:05Z
body: > Do you have any ideas on which IDEs would be a good test bed?

I'm not sure. I think the [vim LSP](https://microsoft.github.io/pyright/#/installation?id=vim) is the most popular (based purely on the quantity of issues I've received), so that might be a good candidate.
author_association: COLLABORATOR
user: login: rchiodo
created_at: 2025-03-11T16:37:13Z
body: > LGTM. It might be worth doing a quick manual test of the CLI with watch mode (--watch). We don't have any tests for this, and it could potentially be affected by some of the code paths you touched in this PR

Okay will do. Thanks for the feedback.
author_association: COLLABORATOR
user: login: rchiodo
created_at: 2025-03-11T17:15:30Z
body: neovim works with the new changes. 
author_association: COLLABORATOR
user: login: rchiodo
created_at: 2025-03-11T17:20:29Z
body: watch mode works too (still produces errors on file change)
author_association: COLLABORATOR
user: login: github-actions[bot]
created_at: 2025-03-11T19:27:53Z
body: According to [mypy_primer](https://github.com/hauntsaninja/mypy_primer), this change doesn't affect type check results on a corpus of open source code. ✅
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-11T19:44:50Z
body: Diff from [mypy_primer](https://github.com/hauntsaninja/mypy_primer), showing the effect of this PR on open source code:
```diff
sympy (https://github.com/sympy/sympy)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:120:18 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:120:18 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:123:18 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:123:18 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:126:9 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:126:9 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:132:9 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer" cannot be assigned to parameter "expr" of type "Expr" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:132:9 - error: Argument of type "Equality | Unknown" cannot be assigned to parameter "expr" of type "Expr" in function "add"
-     Type "Equality | Unknown | Relational | Unequality | Integer" is not assignable to type "Expr"
+     Type "Equality | Unknown" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Equality" is not assignable to "Expr" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:895:12 - error: Operator "/" not supported for types "Equality | Unknown | Relational | Unequality | Integer" and "Literal[2]"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:895:12 - error: Operator "/" not supported for types "Equality | Unknown" and "Literal[2]"
-     Operator "/" not supported for types "Equality" and "Literal[2]"
-     Operator "/" not supported for types "Relational" and "Literal[2]"
-     Operator "/" not supported for types "Unequality" and "Literal[2]" (reportOperatorIssue)
+     Operator "/" not supported for types "Equality" and "Literal[2]" (reportOperatorIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1863:12 - error: Cannot access attribute "combsimp" for class "Relational"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1863:12 - error: Cannot access attribute "combsimp" for class "Unequality"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1911:14 - error: Cannot access attribute "factor" for class "Relational"
-     Attribute "factor" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1911:14 - error: Cannot access attribute "factor" for class "Unequality"
-     Attribute "factor" is unknown (reportAttributeAccessIssue)
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1918:14 - error: Cannot access attribute "combsimp" for class "Relational"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1918:14 - error: Cannot access attribute "combsimp" for class "Unequality"
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1932:12 - error: Cannot access attribute "combsimp" for class "Relational"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1932:12 - error: Cannot access attribute "combsimp" for class "Unequality"
-     Attribute "combsimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
-     Type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" is not assignable to type "ConvertibleToFloat"
+     Type "Equality | Unknown | NaN | Sum" is not assignable to type "ConvertibleToFloat"
-       Type "Relational" is not assignable to type "ConvertibleToFloat"
+       Type "Equality" is not assignable to type "ConvertibleToFloat"
-         "Relational" is not assignable to "str"
+         "Equality" is not assignable to "str"
-         "Relational" is incompatible with protocol "Buffer"
+         "Equality" is incompatible with protocol "Buffer"
-         "Relational" is incompatible with protocol "SupportsFloat"
+         "Equality" is incompatible with protocol "SupportsFloat"
-         "Relational" is incompatible with protocol "SupportsIndex"
+         "Equality" is incompatible with protocol "SupportsIndex"
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:1992:22 - error: Argument of type "Equality | Unknown | NaN | Sum" cannot be assigned to parameter "x" of type "ConvertibleToFloat" in function "__new__"
-     Type "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum" is not assignable to type "ConvertibleToFloat"
+     Type "Equality | Unknown | NaN | Sum" is not assignable to type "ConvertibleToFloat"
-       Type "Relational" is not assignable to type "ConvertibleToFloat"
+       Type "Equality" is not assignable to type "ConvertibleToFloat"
-         "Relational" is not assignable to "str"
+         "Equality" is not assignable to "str"
-         "Relational" is incompatible with protocol "Buffer"
+         "Equality" is incompatible with protocol "Buffer"
-         "Relational" is incompatible with protocol "SupportsFloat"
+         "Equality" is incompatible with protocol "SupportsFloat"
-         "Relational" is incompatible with protocol "SupportsIndex" (reportArgumentType)
+         "Equality" is incompatible with protocol "SupportsIndex" (reportArgumentType)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2074:56 - error: Cannot access attribute "gammasimp" for class "Relational"
-     Attribute "gammasimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2074:56 - error: Cannot access attribute "gammasimp" for class "Unequality"
-     Attribute "gammasimp" is unknown (reportAttributeAccessIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2973:37 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2973:37 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | NaN | Sum"
-     Operator "*" not supported for types "Literal[2]" and "Equality"
-     Operator "*" not supported for types "Literal[2]" and "Relational"
-     Operator "*" not supported for types "Literal[2]" and "Unequality" (reportOperatorIssue)
+     Operator "*" not supported for types "Literal[2]" and "Equality" (reportOperatorIssue)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2978:17 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | Relational | Unequality | Integer | NaN | Sum"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_wester.py:2978:17 - error: Operator "*" not supported for types "Literal[2]" and "Equality | Unknown | NaN | Sum"
-     Operator "*" not supported for types "Literal[2]" and "Equality"
-     Operator "*" not supported for types "Literal[2]" and "Relational"
-     Operator "*" not supported for types "Literal[2]" and "Unequality" (reportOperatorIssue)

... (truncated 60 lines) ...

```
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-11T20:20:26Z
body: Diff from [mypy_primer](https://github.com/hauntsaninja/mypy_primer), showing the effect of this PR on open source code:
```diff
sympy (https://github.com/sympy/sympy)
-   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:120:70 - error: Argument of type "list[Unequality | Unknown | Equality | Relational]" cannot be assigned to parameter "conditions_nonfalse" of type "List[Expr]" in function "add"
+   /tmp/mypy_primer/projects/sympy/sympy/utilities/tests/test_matchpy_connector.py:120:70 - error: Argument of type "list[Unequality | Integer]" cannot be assigned to parameter "conditions_nonfalse" of type "List[Expr]" in function "add"
-     Type "Unequality | Unknown | Equality | Relational" is not assignable to type "Expr"
+     Type "Unequality | Integer" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr"
+       "Unequality" is not assignable to "Expr"
-     Type "Unequality | Unknown | Equality | Relational" is not assignable to type "Expr"
+     Type "Unequality | Integer" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr"
+       "Unequality" is not assignable to "Expr"
-     Type "Unequality | Unknown | Equality | Relational" is not assignable to type "Expr"
+     Type "Unequality | Integer" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr"
+       "Unequality" is not assignable to "Expr"
-     Type "Unequality | Unknown | Equality | Relational" is not assignable to type "Expr"
+     Type "Unequality | Integer" is not assignable to type "Expr"
-       "Relational" is not assignable to "Expr" (reportArgumentType)
+       "Unequality" is not assignable to "Expr" (reportArgumentType)
+   /tmp/mypy_primer/projects/sympy/sympy/vector/implicitregion.py:114:20 - error: Operator "**" not supported for "None" (reportOptionalOperand)
+   /tmp/mypy_primer/projects/sympy/sympy/vector/implicitregion.py:114:28 - error: Operator "*" not supported for types "Literal[4]" and "One | Zero | Expr | Unknown | Any | None"
+     Operator "*" not supported for types "Literal[4]" and "None" (reportOperatorIssue)
+   /tmp/mypy_primer/projects/sympy/sympy/vector/implicitregion.py:503:30 - error: "coeff" is not a known attribute of "None" (reportOptionalMemberAccess)
+   /tmp/mypy_primer/projects/sympy/sympy/vector/implicitregion.py:504:30 - error: "coeff" is not a known attribute of "None" (reportOptionalMemberAccess)
+   /tmp/mypy_primer/projects/sympy/sympy/vector/implicitregion.py:505:30 - error: "coeff" is not a known attribute of "None" (reportOptionalMemberAccess)
- 33229 errors, 72 warnings, 0 informations 
+ 33234 errors, 72 warnings, 0 informations 

```
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-11T20:47:53Z
body: According to [mypy_primer](https://github.com/hauntsaninja/mypy_primer), this change doesn't affect type check results on a corpus of open source code. ✅
author_association: CONTRIBUTOR
user: login: erictraut
created_at: 2025-03-11T20:50:08Z
body: @rchiodo, I just merged this PR by mistake. I thought I was merging a different PR that I was working on. If this PR wasn't ready to merge, I can back it out. Sorry for any inconvenience.
author_association: COLLABORATOR
user: login: rchiodo
created_at: 2025-03-11T20:51:35Z
body: > @rchiodo, I just merged this PR by mistake. I thought I was merging a different PR that I was working on. If this PR wasn't ready to merge, I can back it out. Sorry for any inconvenience.

No worries. I can just submit follow up changes if @debonte has anymore feedback.
author_association: COLLABORATOR
user: login: erictraut
created_at: 2025-03-11T21:09:29Z
body: @rchiodo, with this change merged, I'm now seeing failures in local tests. Are you seeing these failures? Any suggestions for how to fix them?

 FAIL  src/tests/languageServer.test.ts (23.186 s)
  ● Basic language server tests › Diagnostics pull › background thread diagnostics open mode

    Unhandled method test/getOpenFiles

      at handleResponse (node_modules/vscode-jsonrpc/lib/common/connection.js:595:48)
      at handleMessage (node_modules/vscode-jsonrpc/lib/common/connection.js:432:20)
      at Immediate.<anonymous> (node_modules/vscode-jsonrpc/lib/common/connection.js:402:30)

  ● Basic language server tests › Diagnostics pull › Diagnostic severity overrides test

    thrown: "Exceeded timeout of 15000 ms for a test.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      246 |             });
      247 |
    > 248 |             test('Diagnostic severity overrides test', async () => {
          |             ^
      249 |                 const code = `
      250 | // @filename: test.py
      251 | //// def test([|/*marker*/x|]): ...

      at src/tests/languageServer.test.ts:248:13
      at src/tests/languageServer.test.ts:155:9
          at Array.forEach (<anonymous>)
      at src/tests/languageServer.test.ts:154:19
      at Object.<anonymous> (src/tests/languageServer.test.ts:36:1)

<img width="839" alt="image" src="https://github.com/user-attachments/assets/483778e8-900d-423a-9c68-a501e78fdffa" />

author_association: COLLABORATOR
user: login: rchiodo
created_at: 2025-03-11T21:14:43Z
body: Did you build the test server? `npm run test` is supposed to rebuild it. That first error would be exactly that. The test server didn't update

author_association: COLLABORATOR
user: login: erictraut
created_at: 2025-03-11T21:16:37Z
body: Ah, that explains it. I'm in the habit of running `npx jest` to run the tests. Yes, `npm run test` fixed the problem. Thanks!
author_association: COLLABORATOR