Repository: vscode
Issue: Notebook "variable provider" API
Labels: feature-request,api-proposal,notebook-variables

To enable notebook kernels to show variables in a builtin variable viewer for https://github.com/microsoft/vscode/issues/165445, we need an API for variable values to be reported to vscode. There's a lot of similarity between debugger variables and notebook variables, so the API should be compatible with DAP. If debug variables get any exposure in our extension API, it would share types with notebook variables.

Notes
- The UI for this is still under discussion https://github.com/microsoft/vscode/issues/165445
- The current variables view only shows a single flat list of variables, no scopes, this API supports the same
- Hover
  - The vscode debug support extracts the hovered expression in two ways: by using the `EvaluatableExpressionProvider` from an extension, or by using a slightly hacky regex
  - And it evaluates the result in two ways: by calling "evaluate" on the debugger with the expression if the DA supports this, or by searching scopes in the current stackFrame for a variable with a matching name, and following the path in the format `a.b.c`, which is hacky
  - Notebook support can use the `EvaluatableExpressionProvider` but calling a method like `evaluate` is tough because Jupyter can't respond to an evaluate while a cell is running except in a debug session, and we won't start a debug session unless the user is actually debugging
- Inline Values
  - Have discussed showing variable values in a notebook as inline values, same as the debugger's inline values
  - This works via an extension's `InlineValuesProvider` (or again, hacky fallback), and again, looking up a variable with that name in the current stackFrame

Possible future work
- A "watch"-like feature based on evaluating a string could be useful https://github.com/microsoft/vscode/blob/b4fbff3218b1ec6102b44ff4283f523019ceb2f9/src/vs/workbench/contrib/debug/common/debugProtocol.d.ts#L1352
- Set Value https://github.com/microsoft/vscode/blob/b4fbff3218b1ec6102b44ff4283f523019ceb2f9/src/vs/workbench/contrib/debug/common/debugProtocol.d.ts#L1197

```ts
export interface NotebookController {
	/** Set this to attach a variable provider to this controller. */
	variableProvider?: NotebookVariableProvider;
}

interface VariablesResult {
	namedVariableCount: number;
	indexedVariableCount: number;
}

interface NotebookVariableProvider {
	/** VS Code decides when to call the provider due to execution and variable view visibility. But if other things can change variables (interactive output widgets, background tasks...?) the provider needs to signal us. */
	onDidChangeVariables: Event<void>;

	/**
	 * Here's how this works in DAP:
	 * scopesRequest returns a Scope[].
	 * Each Scope has a variablesReference and named/indexed counts.
	 * variablesRequest is called with the Scope's variablesReference, a start index and a count.
	 */

	/** Returns the global list of variables, with count info for paging. VariablesResult is like the DAP Scope. */
	getVariableCounts(): Promise<VariablesResult>;

	/** When variablesReference is undefined, this is requesting global Variables. When a variableReference is passed, it's requesting child props of another Variable. provideVariables is like the DAP variables request. */
	provideVariables(variablesReference: number | undefined, start: number, count: number): Variable[];
}

// excludes memoryReference and evaluateName from DAP
// need "size" property, should that be part of DAP? Part of the type? In a generic "detail" field?
interface Variable {
	/** The variable's name. */
	name: string;
	/** The variable's value.
		This can be a multi-line text, e.g. for a function the body of a function.
		For structured variables (which do not have a simple value), it is recommended to provide a one-line representation of the structured object. This helps to identify the structured object in the collapsed state when its children are not yet visible.
		An empty string can be used if no value should be shown in the UI.
	*/
	value: string;
	/** The type of the variable's value. Shown in the UI when hovering over the value. */
	type?: string;
	/** Properties of a variable that can be used to determine how to render the variable in the UI. */
	presentationHint?: VariablePresentationHint;
	/** If `variablesReference` is > 0, the variable is structured and its children can be retrieved by calling provideVariables with the variablesReference. */
	variablesReference: number;
	/** The number of named child variables.
		This information is used to present the children in a paged UI and fetch them in chunks.
	*/
	namedVariables?: number;
	/** The number of indexed child variables.
		This information is used to present the children in a paged UI and fetch them in chunks.
	*/
	indexedVariables?: number;
}

// TODO- eliminate values not read by vscode, or include for future compat?
// eliminate values that mention breakpoints
interface VariablePresentationHint {
	// VS Code only supports virtual
	/** The kind of variable. Before introducing additional values, try to use the listed values.
		Values:
		'property': Indicates that the object is a property.
		'method': Indicates that the object is a method.
		'class': Indicates that the object is a class.
		'data': Indicates that the object is data.
		'event': Indicates that the object is an event.
		'baseClass': Indicates that the object is a base class.
		'innerClass': Indicates that the object is an inner class.
		'interface': Indicates that the object is an interface.
		'mostDerivedClass': Indicates that the object is the most derived class.
		'virtual': Indicates that the object is virtual, that means it is a synthetic object introduced by the adapter for rendering purposes, e.g. an index range for large arrays.
		'dataBreakpoint': Deprecated: Indicates that a data breakpoint is registered for the object. The `hasDataBreakpoint` attribute should generally be used instead.
		etc.
	*/
	kind?: 'property' | 'method' | 'class' | 'data' | 'event' | 'baseClass' | 'innerClass' | 'interface' | 'mostDerivedClass' | 'virtual' | 'dataBreakpoint' | string;

	// VS Code only supports readOnly
	/** Set of attributes represented as an array of strings. Before introducing additional values, try to use the listed values.
		Values:
		'static': Indicates that the object is static.
		'constant': Indicates that the object is a constant.
		'readOnly': Indicates that the object is read only.
		'rawString': Indicates that the object is a raw string.
		'hasObjectId': Indicates that the object can have an Object ID created for it.
		'canHaveObjectId': Indicates that the object has an Object ID associated with it.
		'hasSideEffects': Indicates that the evaluation had side effects.
		'hasDataBreakpoint': Indicates that the object has its value tracked by a data breakpoint.
		etc.
	*/
	attributes?: ('static' | 'constant' | 'readOnly' | 'rawString' | 'hasObjectId' | 'canHaveObjectId' | 'hasSideEffects' | 'hasDataBreakpoint' | string)[];

	// VS Code only supports internal
	/** Visibility of variable. Before introducing additional values, try to use the listed values.
		Values: 'public', 'private', 'protected', 'internal', 'final', etc.
	*/
	visibility?: 'public' | 'private' | 'protected' | 'internal' | 'final' | string;

	/** If true, clients can present the variable with a UI that supports a specific gesture to trigger its evaluation.
		This mechanism can be used for properties that require executing code when retrieving their value and where the code execution can be expensive and/or produce side-effects. A typical example are properties based on a getter function.
		Please note that in addition to the `lazy` flag, the variable's `variablesReference` is expected to refer to a variable that will provide the value through another `variable` request.
	*/
	lazy?: boolean;
}
```