Introducing the Node.js package analyzer
Tags: javascript, node, showdev
Published: 2021-04-07 15:51:02

# Introduction
I'm proud to present to you the Node.js package analyzer.
{% github tmkn/packageanalyzer no-readme %}
It's a framework designed to easily introspect a Node.js package.

Think of it as *eslint* for your Node.js package/project.

It's also what powered the data behind [npmbomb](https://npmbomb.tmkn.dev/).

# Package Analyzer
While the JS ecosystem is quite mature and diverse, there is no *linter* for a Node.js project. If you want to do a license compliance check, it's a separate tool. If you wan't to check the health of a dependency, that's a separate tool. If you want to check for unneeded polyfills/deprecated packages, that's yet another tool.

The aim of the package analyzer is to provide the framework and toolset to easily answer questions like above within a single tool.

All while being flexible enough to answer any questions surrounding a Node.js package like:
* calculate the number of all dependencies
* find newest/oldest dependency
* find most included dependency
* get weekly downloads
* check for a new version
* release velocity
* etc

> That's why the package analyzer is both an API and a CLI

Another reason why I started this project was security.

Personally I think there's to much trust put into adding dependencies.

Any Node.js package that you install can run a `postinstall` script and [people tried to extract credentials via this way for some time now](https://snyk.io/vuln/npm:radar-cms), yet there is no straight forward way to highlight packages with a `postinstall` script.

I also envision a system that highlights which API's a particular dependency is using. `fs`, `http` etc. or highlights differences between version upgrades: A new sub dependency was added? By whom?

Ideally the package analyzer will be able to answer those questions.


## Architecture
The package analyzer is written in TypeScript so TypeScript types are a first class citizen.

### Package
At the heart of all of it, is the Package class. After you've traversed a package and all of its dependencies you'll get a single instance of the Package class back. Important points of the API are as follows:
```typescript
interface IPackage<T> {
    //each package parent's can be easily accessed
    parent: T | null;

    //vital information is directly accessible
    name: string;
    version: string;
    fullName: string; //`name@version`

    //dependencies are listed here
    directDependencies: T[];

    //convenience functions to iterate over the dependency tree
    visit: (callback: (dependency: T) => void, includeSelf: boolean, start: T) => void;

    ///easily find dependencies
    getPackagesBy: (filter: (pkg: T) => boolean) => T[];
    getPackagesByName: (name: string, version?: string) => T[];

    //access package.json data
    getData(key: string): unknown;

    //access custom data
    getDecoratorData<E extends IDecoratorStatic<any, []>>(decorators: E): DecoratorType<E>;
}
```
For more information about the architecture you can checkout the [Architecture.md](https://github.com/tmkn/packageanalyzer/blob/master/ARCHITECTURE.md) in the GitHub repository.

## CLI
If you install the package analyzer globally:
```
npm install -g @tmkn/packageanalyzer@0.9.4
```
You will get a `pkga` command where you can easily introspect packages.

Some of the things it does currently are as follows:

### Print Metadata
You can use the `analyze` option to print metadata.
If you don't provide a version number it will use the latest release.
```
pkga analyze --package react //use latest version
pkga analyze --package react@16.12.0 //use specific version
```
![pkga analyze](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6f2khzn1an8dd0vbujfb.png)

Use the `--full` option to print additional data like oldest/newest package.
```
pkga analyze --package react --full
```
![pkga analyze full](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l39t0bivms1t7sra6czn.png)

If you want to analyze a local project use the `--folder` option:
```
pkga analyze --folder path/to/your/package.json
```

### Print Dependency Tree
```
pkga tree --package react
```
![pkga tree](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d7wly4vd4ethho3kbiiq.png)

### Print Weekly Downloads
The `downloads` command will print the weekly downloads for a package for NPM
```
pkga downloads --package react
```
![pkga downloads](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rz1cs0hlyu9dkxc6a0lh.png)

### Cyclic Dependencies
Use the `loops` command to print cyclic dependencies in the dependency tree:
```
pkga loops --package webpack@4.46.0
```
![pkga loops image](https://i.imgur.com/gZFx8mA.png)

## API
In addition to the CLI, the Package Analyzer also offers an API.
All the commands in the CLI are done via this API as well.

### Example
Here's how you can use it to list all dependencies of `fastify` that come with built int `TypeScript` support.
Type declarations are either marked via the `types` or `typings` field in the `package.json`, so all we need to do is ask if those fields are set and collect them:
```typescript
const { Visitor, npmOnline, OraLogger } = require("@tmkn/packageanalyzer");

(async () => {
    try {
        const visitor = new Visitor(["fastify"], npmOnline, new OraLogger());
        const pkg = await visitor.visit();

        const matches = new Set();

        pkg.visit(dep => {
            if (dep.getData("types") || dep.getData("typings"))
                matches.add(dep.fullName);
        }, true);

        console.log("Built in TypeScript support:")
        for (const name of matches)
            console.log(name);
    }
    catch (e) {
        console.log(e);
    }
})();
```

```typescript
        const visitor = new Visitor(["fastify"], npmOnline, new OraLogger());
```
First we need to create a `Visitor` that traverses the dependency tree.

The 1st argument is a tuple that specifies the package, if you don't provide a version. e.g.`["fastify", "3.14.1"]` it will default to the latest like here.

The 2nd argument is the `Provider`. Whenever the `Visitor` wants data about a package it will ask the `Provider`. In this case we are asking the NPM registry, but you could also write a `Provider` that gets the information from the filesystem e.g. `node_modules` folder.

The 3rd argument is the logger. All output is routed to this logger.

```typescript
        const pkg = await visitor.visit();
```
Then we simply call the async `visit` function from the `Visitor` to start traversing the dependency tree. We will end up with a single top level `Package` class.

```typescript
        pkg.visit(dep => {
            if (dep.getData("types") || dep.getData("typings"))
                matches.add(dep.fullName);
        }, true);
```
This `Package` class provides utility functions like `visit` to iterate over each dependency and the `getData` method to access the respective `package.json`. So here we iterate over each dependency and check if the respective `package.json` contains entries for `types` or `typings`. If yes, collect `dep.fullName` which is a string formatted to `packagename@version`.

```typescript
        console.log("Built in TypeScript support:")
        for (const name of matches)
            console.log(name);
```
Afterwards we just print out our findings, done.

If you want to know more I recommend to check out the [Architecture.md](https://github.com/tmkn/packageanalyzer/blob/master/ARCHITECTURE.md) in the GitHub repository. But do note that the API is not yet stable and will likely undergo changes as it's a first preview.

## What's next
Since I have a lot of ideas where to take this and since it's a first preview I would really appreciate feedback :)

In the beginning I said to think of it as `eslint` for packages, however defining your own set of checks that you want to run is not yet possible and something that I want work on next.

I'm also thinking about a web interface, that allows to better visually present the results and where you can jump back and forth between packages/reports in an easy manner.

Apart from that, any suggestions are highly welcomed.

For updates you can follow me on [Twitter](https://twitter.com/tmkndev) or follow the project on GitHub: https://github.com/tmkn/packageanalyzer