AI-assisted Drupal development: testing GitHub Copilot
Tags: drupal, ai, vscode, githubcopilot
Published: 2021-07-21 16:29:16

On June 29, 2021 GitHub [announced](https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/) [GitHub Copilot](https://copilot.github.com/), an Artificial Intelligence (AI)-based assistant that helps developers write code.

The tool is currently only available as a [VS Code extension](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot) and access is gradually granted via a [waiting list](https://github.com/features/copilot/signup).

I was fortunate to join the program a few days ago and tested it against Drupal 9.

## How it works

Details can be found on the [GitHub Copilot](https://copilot.github.com/) page. In short:
1. [OpenAI](https://openai.com/) trained an AI model against billions of lines of code stored in public repositories
2. That model is accessible via a service hosted by GitHub
3. As the developer types code in VS Code, the latter sends queries to the GitHub service, which then tries to understand the developer's intent and responds with code suggestions
4. The service improves over time as it gathers information about whether developers accept or reject code suggestions

While this mechanism may give rise to security concerns (snippets of your local, private code will be sent to GitHub), the [FAQ section](https://copilot.github.com/#faqs) and [this page](https://docs.github.com/en/github/copilot/about-github-copilot-telemetry) provide some clarification. In particular:
* Your private code will not be offered as suggestions to others developers
* Only select employees at GitHub, OpenAI and Microsoft will have access to the parts of your code that were sent to the GitHub service

## What else is there

GitHub Copilot is not the first AI-based assistant for developers.

Microsoft released [Visual Studio IntelliCode](https://marketplace.visualstudio.com/items?itemName=VisualStudioExptTeam.vscodeintellicode) (based on [GPT-2](https://en.wikipedia.org/wiki/GPT-2)) in 2018. It [does not support PHP](https://github.com/MicrosoftDocs/intellicode/issues/92) though.

[Tabnine](https://www.tabnine.com/) (available for both [VS Code](https://marketplace.visualstudio.com/items?itemName=TabNine.tabnine-vscode) and [JetBrains](https://plugins.jetbrains.com/plugin/12798-tabnine-ai-code-completion-js-java-python-ts-rust-go-php--more)) is also based on GPT-2. It works OK but seems to have a fairly limited knowledge of Drupal. I could not make it implement a [Drupal hook](https://www.drupal.org/docs/creating-custom-modules/understanding-hooks) for me, for instance.

GitHub Copilot, for its part, supports PHP and is based on Codex, an extension of [GPT-3](https://en.wikipedia.org/wiki/GPT-3) which was trained on a much larger set of code than GPT-2 and so hopefully has a better understanding of Drupal.

## Hello, AI

Let's start by creating a blank PHP file in VS Code and typing some trivial code:

![PHP code with comment that shows intent to compute an average](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rucfwwcn199z6d1m1hk0.png)

Copilot rightly understood from the comment I wrote that I wanted to compute the average of `$ratings` and offered me a line of code that does just that. Then I just have to hit `Tab` to accept the snippet. Neat.

Now, instead of writing a comment, let's define a function `computeAverage()` and see what Copilot offers:

![PHP code with function name that shows intent to compute an average (start writing code)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mnnyq9f2i4xgzwv7g8k8.png)

![PHP code with function name that shows intent to compute an average (all code written)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vunc2kt1875vy85mwuea.png)

I just had to hit `Tab` repeatedly as suggestions showed up to come up with a complete function that computes the average.

Note that only single-line suggestions are offered in PHP. Multi-line suggestions are only supported for Python, JavaScript, TypeScript, Ruby and Go.

## Do you know about Drupal ?

Now let's try Copilot against a fresh Drupal project and see if it is of any help. The model is claimed to have been trained on billions of lines of public code, so hopefully that includes Drupal core and contrib.

```sh
$ composer create-project drupal/recommended-project drupal 9.2
$ code drupal
```

Let's create a file (and its parent directories) at `web/modules/custom/mymodule/src/Form/MyForm.php`, insert a comment at the top describing what kind of Drupal form we want to build and see if any suggestion shows up:

![Copilot suggests namespace that complies with Drupal standards](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/slcvnbwll092fxkn0yax.png)

After typing `namespace`, Copilot suggests a properly formatted [PSR-4 Drupal namespace](https://www.drupal.org/docs/develop/standards/psr-4-namespaces-and-autoloading-in-drupal-8). That is a good sign. 

Let's accept the suggested namespace, and keep going by typing `class`:

![Copilot suggests class that complies with Drupal standards](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mtcczmw1di12mnfo36k2.png)

Copilot suggests a proper class name and extends the right parent class. It sounds like it knows about Drupal. Awesome !

## Can you write that Drupal module for me ?

Let's keep hitting `Tab` as suggestions show up and insert a few PHP comments here and there. Copilot ends up writing the entire class:

![Copilot wrote the whole form, with a few glitches](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t6zrjzmruvhgrdntslx9.png)

It looks pretty good:
* The AI implemented exactly the right methods (`getFormId()`, `buildForm()`, `validateForm()`, `submitForm()`) albeit with deprecated signatures
* The contents of each method makes sense
* The [Drupal coding standards](https://www.drupal.org/docs/develop/standards/coding-standards) are generally observed, with the notable exception of the array syntax
* The field titles were wrapped in a `t()` function, which again demonstrates some Drupal-specific knowledge
* The Email field was defined with type `email` instead of `textfield`

A few mishaps:
* Arrays were formatted using the [long syntax](https://www.php.net/manual/en/language.types.array.php#language.types.array.syntax). That policy was [changed](https://www.drupal.org/node/2135291) several years ago. Maybe Copilot was trained against projects that still use the long syntax and/or is not aware that I am writing code for Drupal 9 instead of Drupal 7. That being said, a well-configured [phpcs extension](https://marketplace.visualstudio.com/items?itemName=ikappas.phpcs) would quickly bring this to the developer's attention.
* Function `drupal_set_message()` was introduced even though it was [deprecated](https://www.drupal.org/node/2774931) a few years ago. Again, Copilot may not be aware that I am writing code specifically for Drupal 9. The [drupal-check extension](https://marketplace.visualstudio.com/items?itemName=bbeversdorf.drupal-check) would likely catch this.
* Both fields were made mandatory in `buildForm()` even though the comment I typed suggested that I wanted only the Name field to be required.
* The methods were implemented with deprecated signatures (`$form_state` is supposed to be a `FormStateInterface` object, not an array). The [change](https://www.drupal.org/node/2310411) happened several years ago.

Overall, the suggestions make sense and look rather helpful.

## Trying to fool the AI

Let's create a file at `web/modules/custom/mymodule/mymodule.module` and prompt Copilot to offer code that displays user-entered content:

![Copilot suggests code that includes protection against XSS attacks](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5q9w5h85k19n5qbubp2o.png)

First off, Copilot retrieved the argument from the URL *the Drupal way*, but using the Drupal 7 function `arg()` which was [deprecated](https://www.drupal.org/node/2274705) in Drupal 8.

Still, the code makes use of a `t()` function and a safe `@` placeholder, which brings protection against [XSS attacks](https://en.wikipedia.org/wiki/Cross-site_scripting). The code looks secure.

Now let's do something similar with a SQL query:

![Copilot suggests code that includes protection against SQL injections, but with deprecated code (start of code)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p3kl6hkmd01kbr1ndqh6.png)

![Copilot suggests code that includes protection against SQL injections, but with deprecated code (code complete)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ftdxy2tkmkzismd36hnh.png)

The suggested code makes use of the [database API](https://www.drupal.org/docs/drupal-apis/database-api), which includes a protection against [SQL injections](https://en.wikipedia.org/wiki/SQL_injection). Again, the code looks secure. However, it is based on the [deprecated](https://www.drupal.org/node/2993033) function `db_select()`. Let's give Copilot a hint:

![Copilot suggests code that includes protection against SQL injections, and is not deprecated (start of code)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1oehozxww5r818ucr2hi.png)

![Copilot suggests code that includes protection against SQL injections, and is not deprecated (code complete)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/116xjn1nrlkb9m06k6p7.png)

This time Copilot suggested code that makes use of the up-to-date function `\Drupal::entityQuery()`.

## Can we talk ?

Copilot happens to offer suggestions in a large set of programming languages, but also in human languages. By formatting a plain text file into what looks like a conversation, it managed to answer my questions:

![Conversation with Copilot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h4q0id8rjll4gqa7b0hn.png)

Someone took this to the next level by making Copilot write a fake paper in LaTeX ([watch here](https://www.youtube.com/watch?v=V1-hRy8EgNE)). While that is not its primary purpose, and the answers don't always make sense, I found this side effect rather intriguing.

## Conclusion

GitHub Copilot looks valuable for Drupal development, as long as you are comfortable sharing parts of your private code with GitHub.

Support for multi-line suggestions in PHP would definitely add more value, although the tool remains relevant even with single-line suggestions.

While it does not seem to be fully aware of Drupal versions and deprecated code, it was able to generate code that for the most part followed coding standards, was secure, and mostly worked.

While GitHub Copilot does not seem to be able to [replace](https://www.economist.com/science-and-technology/2021/07/07/ai-is-transforming-the-coding-of-computer-programs) Drupal developers at this point, it certainly has the potential to make them more productive.