How to write your own language plugin for IDEA (part 2)
Tags: java, kotlin, idea, jetbrains
Published: 2022-04-26 18:22:23


Disclaimer: I don't work for JetBrains, so there may be and most likely there will be inaccuracies and errors in the article and code.

In the [previous article](https://dev.to/pyltsinm/how-to-write-your-own-language-plugin-for-idea-part-1-24ea-temp-slug-5129265), I showed the process of creating a framework for a language plugin. Well-known plugins for [Java](https://github.com/JetBrains/intellij-community), [go](https://github.com/go-lang-plugin-org/go-lang-idea-plugin), [Frege](https://github.com/IntelliJ-Frege) were used as examples. There are also examples from the plugin for the Monkey language, which I developed while I was figuring out how everything works. Since my goal was not to cover everything, the plugin covers a limited subset of the language. The interpreter for Monkey can be found [here](https://github.com/pyltsin/monkey-source).

#Formatting
[Documentation](https://plugins.jetbrains.com/docs/intellij/code-formatting.html), examples from [go-plugin](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/tree/master/src/com/goide/formatter), [monkey](https://github.com/pyltsin/monkey-plugin/blob/main/src/main/kotlin/com/github/pyltsin/monkeyplugin/formatter/MonkeyFormattingModelBuilder.kt).
As always, everything starts from an extension point.

```xml
<lang.formatter language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.formatter.MonkeyFormattingModelBuilder"/>
```
A class must implement the next interface

```java
public interface FormattingModelBuilder {
@NotNull 
FormattingModel createModel(@NotNull FormattingContext formattingContext);

@Nullable 
TextRange getRangeAffectingIndent(PsiFile file, int offset,
                                  ASTNode elementAtOffset) ;
}
```

The most important method is the first one. It returns a formatting model, which is built on formatting blocks. To simplify it, you can use  `FormattingModelProvider.createFormattingModelForPsiFile ` method

Let's take a look at what a formatting block is. In IDEA, a formatting block is represented as an interface `com.intellij.formatting.Block`. It is some range of text (often corresponding to some PSI element) which is formatted according to some rules. The formatting blocks are nested into each other and create a tree.

```java
public interface Block {

  @NotNull
  TextRange getTextRange();

  @NotNull
  List<Block> getSubBlocks();

  @Nullable
  Wrap getWrap();

  @Nullable
  Indent getIndent();

  @Nullable
  Alignment getAlignment();

  @Nullable
  Spacing getSpacing(@Nullable Block child1, @NotNull Block child2);

  @NotNull
  ChildAttributes getChildAttributes(final int newChildIndex);

/// some methods are skipped
}
```

If you want to visualize this tree you can use PSI-Viewer 
 (Tools->View PSI Structure, tab "Block Structure")
![Visualization formatting blocks](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o8xnh5et8g6nljt6wh4d.png)

To simplify the implementation, you can use the `AbstractBlock` class.

`getSpacing` - determines the number of spaces or line breaks between the specified child elements. To simplify the implementation of this logic, you can use the `com.intellij.formatting.SpacingBuilder` class, which provides a convenient API to describe rules.

```java
//an extraction from plugin for Golang
public Spacing getSpacing(@Nullable Block child1, @NotNull Block child2) {
return new SpacingBuilder(settings, GoLanguage.INSTANCE)
      .before(COMMA).spaceIf(false)
      .after(COMMA).spaceIf(true)
      .betweenInside(SEMICOLON, SEMICOLON, FOR_CLAUSE).spaces(1)
      .before(SEMICOLON).spaceIf(false)
      .after(SEMICOLON).spaceIf(true)
      .beforeInside(DOT, IMPORT_SPEC).none()
      .afterInside(DOT, IMPORT_SPEC).spaces(1)
      //and more rules
      .getSpacing(this, child1, child2);
}
```

#Structure view

[Documentation](https://plugins.jetbrains.com/docs/intellij/structure-view.html). Examples of implementation in [plugin-go](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/tree/GoStructureViewFactory.java), [Monkey](https://github.com/pyltsin/monkey-plugin/blob/main/src/main/kotlin/com/github/pyltsin/monkeyplugin/editor/MonkeyStructureViewFactory.kt).

In this section we will talk about filling this panel:
![Structure view panel](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a95josqqdn6nk17vbsel.png)

The next extension point is responsible for that
```xml
<lang.psiStructureViewFactory language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.editor.MonkeyStructureViewFactory"/>
```
Your class must implement the next interface:
```java
@FunctionalInterface
public interface PsiStructureViewFactory {
  @Nullable
  StructureViewBuilder getStructureViewBuilder(@NotNull PsiFile psiFile);
}
```
For implementation `StructureViewBuilder` you can also  use prepared classes: `com.intellij.ide.structureView.TreeBasedStructureViewBuilder`, `com.intellij.ide.structureView.StructureViewModelBase` and `com.intellij.ide.structureView.impl.common.PsiTreeElementBase`.

Working with `PsiTreeElementBase` is similar to working with formatting blocks.

#Caches, indexes, stub and goto

##Caches

Let's start with the caches. IDEA asks extensions several times. If the extension is doing some time-consuming work, then the best solution is to cache the result of this work. IDEA provides many wrappers for this. For example, this is how the type of Go expression is calculated in the plugin-go:

```java
  @Nullable
  public static GoType getGoType(@NotNull GoExpression o, @Nullable ResolveState context) {
    return RecursionManager.doPreventingRecursion(o, true, () -> {
      if (context != null) return unwrapParType(o, context);
      return CachedValuesManager.getCachedValue(o, () -> CachedValueProvider.Result
        .create(unwrapParType(o, createContextOnElement(o)), PsiModificationTracker.MODIFICATION_COUNT));
    });
  }
```

2 different managers are used here. The first is the `CachedValuesManager`, which caches the result for the psi element, and the second is the `Recursion Manager`, which helps to prevent infinite recursion and `StackOverflowError`. There is also a specialized cache `com.intellij.psi.impl.source.resolve.ResolveCache`, which is used for resolving elements (this will be specified below).

##Indexes

![About indexing from Reddit](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5tbqgf6qmcxnxf8xe2gw.png)
 
[Documentation](https://plugins.jetbrains.com/docs/intellij/indexing-and-psi-stubs.html#indexes). 

We all know how IDEA likes to index everything. Let's see what it is and how it can be used.

Indexes in IDEA provide the opportunity to quickly find a necessary file or a psi element (for example, by the name of the annotation, you can find all the places where it is used).

You can view existing indexes using the [Index viewer](https://plugins.jetbrains.com/plugin/13029-index-viewer) plugin.

IDEA supports two types of indexes - File-based and Stub. File-based indexes work with the contents of a file, Stub indexes work with Stub-tree, which is built on the basis of PSI-tree.

##File-based indexes

An example of use can be found in the [hackforce plugin](https://github.com/carymrobbins/intellij-haskforce)

We, as usual, start from the extension point

```xml
<fileBasedIndex implementation="com.haskforce.index.HaskellModuleIndex"/>
```

Class must extend `FileBasedIndexExtension`. To get result of indexing you can use `FileBasedIndex.getInstance()`

The hack force plugin, for example, uses this type of index to get all the files within the module:

```java
@NotNull
public static Collection<VirtualFile> getVirtualFilesByModuleName(@NotNull String moduleName, @NotNull GlobalSearchScope searchScope) {
  return FileBasedIndex.getInstance()
    .getContainingFiles(HASKELL_MODULE_INDEX, moduleName, searchScope);
}
```

##Stub indexes

It seems to me that this type of indexes is used more often because it allows you to search through PSI elements (or rather through stub, which represents the required part of the psi tree). Stubs are used only for named psi elements (which implement the `PsiNamedElement` interface). They will be described in more detail in the Reference section

To declare a new index, the following extension point is used:

```xml
<stubIndex implementation=
"com.github.pyltsin.monkeyplugin.stubs.MonkeyVarNameIndex"/>
```

Your class must implement `StubIndexExtension`.
Example from the plugin for Monkey:
```kotlin
class MonkeyVarNameIndex : StringStubIndexExtension<MonkeyNamedElement>() {
    override fun getVersion(): Int {
        return super.getVersion() + VERSION
    }

    override fun getKey(): StubIndexKey<String, MonkeyNamedElement> {
        return KEY
    }

    companion object {
        val KEY: StubIndexKey<String, MonkeyNamedElement> =
            StubIndexKey.createIndexKey("monkey.var.name")
        const val VERSION = 0
    }
}
```

Examples from [go-plugin](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/stubs/index/GoAllPublicNamesIndex.java), [frege](https://github.com/IntelliJ-Frege/intellij-frege/tree/master/src/main/kotlin/com/plugin/frege/stubs/index).

Now we need to teach IDEA how to create a tree of Stubs and save the necessary elements under the desired index.

For each element type that we want to save as a Stub, we create a Stub definition. In this case, the root of all Stubs should be `FileStub`.

Example from the plugin for Monkey:

```kotlin
class MonkeyFileStub(file: MonkeyFile?) : PsiFileStubImpl<MonkeyFile>(file)

class MonkeyVarDefinitionStub : NamedStubBase<MonkeyVarDefinition> {
    constructor(parent: StubElement<*>?, elementType: IStubElementType<*, *>, name: StringRef?) : super(
        parent,
        elementType,
        name
    )

    constructor(parent: StubElement<*>?, elementType: IStubElementType<*, *>, name: String?) : super(
        parent,
        elementType,
        name
    )
}
```

Examples from go-plugin ([file](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/stubs/GoFileStub.java), [element](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/stubs/GoFieldDefinitionStub.java)), Frege ([file](https://github.com/IntelliJ-Frege/intellij-frege/blob/master/src/main/kotlin/com/plugin/frege/stubs/FregeFileStub.kt), [element](https://github.com/IntelliJ-Frege/intellij-frege/blob/master/src/main/kotlin/com/plugin/frege/stubs/FregeClassStub.kt))

The next step is to create a description of the element type for each Stub. (For automatically generated PSI elements with  Grammar-Kit plugin, descriptions of each type of element are created automatically following `elementTypeHolderClass` and `elementTypeClass` parameters). The `ElementType` for the file must extend `IStubFileElementType`, for the element - `IStubElementType`.

`IStubElementType` requires the implementation of the following methods:

```java
@NotNull
String getExternalId();

void serialize(@NotNull T stub, @NotNull StubOutputStream dataStream) throws IOException;

@NotNull
T deserialize(@NotNull StubInputStream dataStream, P parentStub) throws IOException;

void indexStub(@NotNull T stub, @NotNull IndexSink sink);

PsiT createPsi(@NotNull StubT stub);

@NotNull StubT createStub(@NotNull PsiT psi, StubElement<?> parentStub);

shouldCreateStub(ASTNode node)
```

How to index Stubs is specified in the index Sub method. For example, in Monkey I used this implementation:

```kotlin
    override fun indexStub(stub: S, sink: IndexSink) {
        val name = stub.name
        if (name != null) {
            sink.occurrence(MonkeyVarNameIndex.KEY, name)
        }
    }
```

The implementation of other methods can be found in the examples - [plugin for Monkey](https://github.com/pyltsin/monkey-plugin/blob/main/src/main/kotlin/com/github/pyltsin/monkeyplugin/stubs/types/MonkeyVarDefinitionStubElementType.kt), [go-plugin](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/stubs/types/GoMethodDeclarationStubElementType.java), [Frege](https://github.com/IntelliJ-Frege/intellij-frege/blob/master/src/main/kotlin/com/plugin/frege/stubs/types/FregeMethodElementType.kt)

Now our stubs need to be connected to the parser. This can be done in 2 steps.

- Step 1: define your element type factory for which we have made `IStubElementType`, for example as

```kotlin
object MonkeyElementTypeFactory {
    @JvmStatic
    fun factory(name: String): IElementType {
        if (name == "VAR_DEFINITION") return MonkeyVarDefinitionStubElementType(name)
        throw RuntimeException("Unknown element type: $name")
    }
}
```

and specify in `bnf` file that it should be used for some PSI elements:

```xml
elementTypeFactory("var_definition")=
"com.github.pyltsin.monkeyplugin.psi.impl.MonkeyElementTypeFactory.factory"
```

- Step 2. Specify that these PSI elements should extend `StubBasedPsiElementBase`

Let's consider everything together. During indexing, `StubBasedPsiElementBase` is created, then a Stub is created using `IStubElementType.createStub`. This stub can be serialized and a reference to it is saved in the index(`indexStub`).

Client code which works with Stubs should call only those methods that have enough stored information to execute. Therefore, it is necessary to include in a stub all the information that may be needed later in the analysis. To get the PSI element, you can call the `getNode()` method, but it is expensive because it requires file parsing.

An example of saving information can be found in `com.intellij.psi.impl.java.stubbs.impl.PsiAnnotationStubImpl#getPsiElement`, which uses text from ASTNode.

###Using stub indexes

Indexes are widely used for go-to functions. For example, they are used for this panel:
![Goto panel](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kuzhq4cx878sb07siv5b.png)
 
Simplified go-to implementation for symbols based on stub indexes:

- plugin.xml
```xml
<gotoSymbolContributor implementation=
"com.github.pyltsin.monkeyplugin.usages.MonkeySymbolContributor"/>
```

- MonkeySymbolContributor:

```java
class MonkeySymbolContributor : ChooseByNameContributorEx {
    private val myIndexKeys = listOf(MonkeyVarNameIndex.KEY)
    override fun processNames(
        processor: Processor<in String>,
        scope: GlobalSearchScope,
        filter: IdFilter?
    ) {
        for (key in myIndexKeys) {
            ProgressManager.checkCanceled()
            StubIndex.getInstance().processAllKeys(
                key,
                processor,
                scope,
                filter
            )
        }
    }

    override fun processElementsWithName(
        name: String,
        processor: Processor<in NavigationItem>,
        parameters: FindSymbolParameters
    ) {
        for (key in myIndexKeys) {
            ProgressManager.checkCanceled()
            StubIndex.getInstance().processElements(
                key,
                name,
                parameters.project,
                parameters.searchScope,
                parameters.idFilter,
                MonkeyNamedElement::class.java,
                processor
            )
        }
    }
}
```

Many custom plugins also use indexes widely. For example, [Request mapper](https://plugins.jetbrains.com/plugin/9567-request-mapper) (currently not supported, since the same functionality appeared in IDEA), which helps to search for REST method declaration points
![Request mapper](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/irals054zd5nxdzgvtt8.png)

![Example from IDEA](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qx1ixilm2amunrhp1neb.png)
 
 Request Mapper uses this code under the hood:
```java
//JavaAnnotationIndex - the usual Subindexbindex for Java for annotations
JavaAnnotationIndex
            .getInstance()
            .get(annotationName, project, GlobalSearchScope.projectScope(project))
            .asSequence()
```

###References

[Documentation](https://plugins.jetbrains.com/docs/intellij/declarations-and-references.html)

At the moment, the API is being changed, so there may be inaccuracies.

![Example of using references](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nl98k46er8llmqojzkyt.gif)
 
References create links between elements. When you press `Ctrl+B`, you will go to the element to which this link refers. By pressing Ctrl+B again, you will see all the elements that link to this element.

Only the element that defines a name should implement `PsiNamedElement` (or better `PsiNameIdentifierOwner`, you can see their use in the Rename section)


```java
@JvmStatic
fun setName(expr: MonkeySimpleRefExpr, name: String): PsiElement {
	val e: PsiElement = MonkeyElementTextFactory.createStatementFromText(expr.project, "$name + 1")
	//newLetExpr must implement PsiNamedElement
  val newLetExpr = PsiTreeUtil.findChildOfType(e, MonkeySimpleRefExpr::class.java)
	if (newLetExpr != null) {
		expr.replace(newLetExpr) 
	}
	return expr
}
```


In order for the PSI element to provide a link, you need to implement the methods


```java
PsiReference getReference();

PsiReference @NotNull [] getReferences();

@Experimental
default @NotNull Iterable<? extends @NotNull PsiSymbolReference> getOwnReferences() {
	return Collections.emptyList();
}
```

To simplify the implementation of the `PsiReference interface`, you can use the `PsiReferenceBase`. You have to implement the method `PsiElement resolve()` or `ResolveResult [] multiResolve(boolean incompleteCode)`, which return the referenced elements. When implementing this method, it makes sense to use a specialized cache:

```java
override fun multiResolve(incompleteCode: Boolean): Array<ResolveResult> {
    return ResolveCache.getInstance(psiElement.project).resolveWithCaching(
        this, { referenceBase, _ ->
            referenceBase.resolveInner(false)
                .map { PsiElementResolveResult(it) }
                .toTypedArray()
        },
        true, false
    )
}
```

After the implementation of this part, IDEA will already be able to provide navigation to the referenced element and back.


###Rename
[Documentation](https://plugins.jetbrains.com/docs/intellij/rename-refactoring.html)

Renaming is one of the most popular refactorings (Shift+F6)

![Renaming](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bay546db7ai9hs4tng2s.gif)

2 extension points are responsible for that:

```xml
<lang.refactoringSupport language="Monkey"
                         implementationClass="com.github.pyltsin.monkeyplugin.refactor.MonkeyRefactoringSupportProvider"/>
<renameInputValidator implementation="com.github.pyltsin.monkeyplugin.refactor.MonkeyRenameInputValidator"/>
```

`renameInputValidator` must implement `RenameInputValidator`.

`refactoringSupport` must extend `RefactoringSupportProvider`. 

This class also contains methods to support other types of refactorings. At the moment we are interested in a method that indicates whether in-place editing is supported.

```java
  public boolean isMemberInplaceRenameAvailable(@NotNull PsiElement element, @Nullable PsiElement context) {
    return false;
  }
```

Now it is required to implement renaming methods. As mentioned before, the PSI element that defines the name should implement the `PsiNamedElement` interface.

```java
public interface PsiNamedElement extends PsiElement {
  String getName();
  PsiElement setName(@NlsSafe @NotNull String name)
  throws IncorrectOperationException;
}
```

We are interested in `setName` method here. One of the easiest ways to implement this method is to create a new PSI element from text, like this

```kotlin
private fun createFileFromText(project: Project, text: String): MonkeyFile {
	return PsiFileFactory.getInstance(project)
		.createFileFromText("A.monkey", MonkeyLanguage.INSTANCE, text) as MonkeyFile
}
```

And replace the element

```kotlin
expr.replace(newLetExpr)
```

It remains to implement the renaming of elements that refer to our named PSI element. To do this, you need to implement the method from `PsiReference`.

```java
PsiElement handleElementRename(@NotNull String newElementName) 
throws IncorrectOperationException;
```

or you can use `com.intellij.psi.PsiReferenceBase`.


###Markers

IDEA widely uses hints in the form of markers
![An example of a marker](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t99xx2n25djskxsw16w1.png)
 
Examples from [go-plugin](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/marker/GoRecursiveCallMarkerProvider.java), [Frege](https://github.com/IntelliJ-Frege/intellij-frege/blob/master/src/main/kotlin/com/plugin/frege/linemarker/related/FregeRelatedItemLineMarkerProvider.kt), [monkey](https://github.com/pyltsin/monkey-plugin/blob/main/src/main/kotlin/com/github/pyltsin/monkeyplugin/editor/MonkeyFunctionLineMarkerProvider.kt)

The required extension point:

```xml
<codeInsight.lineMarkerProvider language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.editor.MonkeyFunctionLineMarkerProvider"/>
```

The file must implement `LineMarkerProvider` interface, whose methods return `LineMarkerInfo` object. Please note that markers should be linked only to the leaves of the PSI-tree.

###Autocompletion

I think everyone who uses IDEA likes how it works with hints. It is very difficult to write a good autocompletion mechanism. But at the same time, you can implement some autocompletion relatively quickly, for example, these:
![Autocompletion with Monkey](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zz8kut9vlutdnlqghnfc.gif)

This completion is implemented with `PsiReference.getVariants` method, which returns all visible suitable values. Filtering by characters is performed by IDEA itself.

For more complex cases, you can use the extension point:

```xml
<completion.contributor language="Monkey"                         implementationClass="com.github.pyltsin.monkeyplugin.completion.MonkeyKeywordCompletionContributor"/>
```

Your class must implement `CompletionContributor` abstract class.

[Documentation](https://plugins.jetbrains.com/docs/intellij/code-completion.html). Examples of implementation from [go-plugin](https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/src/com/goide/completion/GoKeywordCompletionContributor.java), [frege](https://github.com/IntelliJ-Frege/intellij-frege/blob/master/src/main/kotlin/com/plugin/frege/completion/FregeKeywordCompletionContributor.kt), [monkey](https://github.com/pyltsin/monkey-plugin/blob/main/src/main/kotlin/com/github/pyltsin/monkeyplugin/completion/MonkeyKeywordCompletionContributor.kt)


###Testing
[Documentation](https://plugins.jetbrains.com/docs/intellij/writing-tests-for-plugins.html)

In IDEA, tests are often several files with the source code of the language that show the state BEFORE and AFTER the action. `BasePlatformTestCase` class is usually used for it. Usually tests look like this:
```java
myFixture.configureByFiles("RenameTestData.monkey")
myFixture.renameElementAtCaret("test")
myFixture.checkResultByFile("RenameTestData.monkey", "RenameTestDataAfter.monkey", false)
```
where `RenameTestData.monkey` and `RenameTestDataAfter.monkey` are source code files before and after renaming.

#The conclusion
This is the end of the story about creating a language plugin for IDEA. Good luck in setting up your IDE for yourself!