Tool installation and initial configuration - Part 3 of Implementing a RESTful API on AWS
Tags: aws, serverless, npm, vscode
Published: 2021-04-20 18:36:15

Quick recap: this is a soup to nuts series covering implementing a full-featured REST API on AWS from the perspective of a long-time infrastructure engineer with a historical preference for Java-based solutions.

# Table of contents
1. [Preliminaries - Unavoidable Drawbacks](#introduction)
    1. [Dependency Creep](#creep)
    * [Disparate Licensing Terms] (#licensing)
    * [Goodbye Non-corporate Open-Source, Freeware Build Tools] (#freeware)
* [Tool Installation](#tools)
    1. [Node.js - Runtime Engine](#nodejs)
    1. [NPM - Build Automation + Package Manager](#npm)
    1. [VSCode - (Sorta) Integrated Development Environment](#vscode)
    1. [Serverless Framework - Vendor-Agnostic Cloud CLI](#serverless)
* [Conclusion](#conclusion)

---

##<a name="introduction"></a>Preliminaries - Unavoidable Drawbacks
I come from a Eclipse/Maven/Apache-based world of:
1. structured development tools;
2. version control;
3. debuggers; and
4. distrust of libraries that pull in dependencies from all over the Internet.  

In moving to a Javascript-based solution, we're going to have to accept a number of compromises on these fronts.

###<a name="creep"></a>Dependency Creep
For example, when it comes to dependencies, Javascript packages have a tendency to create cascading dependency creep, raising the question of: do I REALLY trust not only the library I am incorporating into my code, but also the cascading libraries that it relies on? Moreover, what license terms have been tacked onto any of these various libraries?  

This has always been one of the benefits of Java-based development and the various standards-based libraries.  I could have some naive-yet-reasonable expectation that anything bundled by Apache/Sun/Oracle/BEA/RedHat/Spring would be covered by specific licensing terms (let's ignore everything after Java 1.8/8) and wouldn't include some chunk of malware pulled in by a random home-brewed regex library.  

Unfortunately, we are about to move into a world where incorporating 4-5 libraries results in 150+ random sub-projects being introduced into your code.

For example, one of the major Javascript testing frameworks--Jest--lists 3 runtime library dependencies.  These then rely on a number of random dependencies, which rely on more random dependencies, etc., etc., until you find yourself including a package called "path-exists"--which I am sure is a wonderful library, but nevertheless is just one more of the 800+ modules that are eventually pulled into my codebase by my handful of initial dependencies.

There [are tools out there to help deal with this] (https://docs.npmjs.com/cli/v7/commands/npm-audit), but a new developer should be aware that this is going to be one of the drawbacks of this stack.

###<a name="licensing"></a>Disparate Licensing Terms
Aside from the security implications, each of these is a library you will eventually need to check from a licensing standpoint.  Although [there are yet again tools to aide you in this task](https://dev.to/connorbode/checking-the-licenses-used-by-your-npm-dependencies-58bi ), it can be time-consuming (albeit nowhere near as time-consuming as writing your own code in place of these libraries).

###<a name="freeware"></a>Goodbye Non-corporate Open-Source, Freeware Build Tools
Finally, one last point before moving onto setting up our build environment: we're leaving behind the open-source build environment of the 1980s-2000s and entering the world of corporate-controlled open-source.

For example, the two primary package managers used with Javascript modules are [npm](https://www.npmjs.com) and [yarn](https://yarnpkg.com).  Although both of these products walk and talk like Apache's [Maven] (http://maven.apache.org), NPM is part of [Microsoft's Github subsidiary] (https://www.npmjs.com/about), while Yarn (like Jest) originates with [Facebook] (https://engineering.fb.com/2016/10/11/web/yarn-a-new-package-manager-for-javascript/).  Likewise, when it comes to IDEs, chances are you are going to be using [Microsoft Visual Studio] (https://visualstudio.microsoft.com)--goodbye [Eclipse Software Foundation] (https://www.eclipse.org).

Obviously, being corporate projects doesn't necessarily make them bad (see [Redhat's Wildfly] (https://www.wildfly.org/about/)), but those of us who have been disappointed with the evolution of Java after Oracle's acquisition of Sun should be forgiven for wanting to see their build tools originate from an entity like Apache.

Fortunately, [Node.js] (https://nodejs.org/en/) itself--our back-end runtime environment--is part of the [OpenJS Foundation] (https://openjsf.org/projects/).

---

##<a name="tools"></a>Tool Installation
All of the following instructions assume you are running MacOS.  Also, although I prefer installing most everything inside of a Docker image for security and consistency, we are going to skip that as it adds too much complexity for this tutorial.

---

###<a name="nodejs"></a>Node.js - Runtime Engine
First, we'll need to install Node.js.  Node.js is the back-end runtime engine for our Javascript code, somewhat like the actual `java` command.  

Check to see if it is installed.  We want at least version 12, preferably version 14.  In early 2021, AWS claimed it added support for version 14, but as of February 2021, certain features, like [nullish coalescing] (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_operator), seemed to be broken.  Either way, check to see if it is installed:

```shell
> node --version
v15.2.1
```

If you get:

```shell
> node --version
-bash: node: command not found
```
it is available [available here](https://nodejs.org/en/download/).

---

###<a name="npm"></a>NPM - Build Automation + Package Manager
In the Java world, we relied on build tools like [Maven](http://maven.apache.org) to handle our builds and package management (or, to really date myself, [Ant build.xml files](https://ant.apache.org) and [Make Makefiles] (https://www.gnu.org/software/make/)).  

With Javascript, it is a two horse race between `npm` and `yarn`. For all intents and purposes, they are interchangeable. Because I generally avoid Facebook, we will be using `npm` (and yes, I will later hypocritically recommend using `jest` over `mocha` as a testing framework). Also, `npm` is included with Node.js.

You can verify npm is installed:
```shell
> npm --version
v7.0.8
```

Whereas maven uses POM manifest files, `npm` and `yarn` use a JSON-based `package.json` manifest file to track information about your dependencies and build.  As should be self-evident from the filename, Javascript tools tend to use JSON (or YAML) instead of the more structured (and verbose) XML.  

Example package.json file:

```json
{
  "name": "my app",
  "version": "1.0.0",
  "description": "This is my app",
  "main": "index.js",
  "scripts": {
    "lint": "tslint -p tsconfig.json -c tslint.json",
    "local": "serverless offline",
    "deploy": "serverless deploy",
    "test": "jest",
    "coverage": "jest --coverage"
  },
  "devDependencies": {
    "@babel/core": "^7.13.10",
    "@babel/preset-env": "^7.13.12",
    "@babel/preset-typescript": "^7.13.0",
    "@shelf/jest-dynamodb": "github:shelfio/jest-dynamodb",
    "@types/aws-lambda": "^8.10.51",
    "@types/aws-sdk": "^2.7.0",
    "@types/jest": "^26.0.21",
    "@types/node": "^14.0.23",
    "@types/uuid": "^8.3.0",
    "babel-jest": "^26.6.3",
    "jest": "^26.6.3",
    "serverless": "^2.30.3",
    "serverless-offline": "^6.8.0",
    "serverless-plugin-typescript": "^1.1.9",
    "typescript": "^3.8.3"
  },
  "dependencies": {
    "dynamoose": "^2.7.3",
    "uuid": "^8.3.2"
  }
}
```

Moreover, because the dependency creep inherent in Javascript modules, `npm` and `yarn` users discovered it can be difficult to ensure the exact versions they are relying on in dependencies of dependencies (`package.json` only lists your direct dependencies).  

To solve this problem, `npm` and `yarn` litter your directory with yet another file: `package-lock.json`.  This file endeavors to list the exact version of every dependency in your dependency tree.

The ~30 line `package.json` file listed above results in a 33,000+ line `package-lock.json` along the lines of:

```json
{
.
.
.
    "node_modules/@babel/core/node_modules/ms": {
      "version": "2.1.2",
      "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
      "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==",
      "dev": true
    },
    "node_modules/@babel/core/node_modules/semver": {
      "version": "6.3.0",
      "resolved": "https://registry.npmjs.org/semver/-/semver-6.3.0.tgz",
      "integrity": "sha512-b39TBaTSfV6yBrapU89p5fKekE2m/NwnDocOVruQFS1/veMgdzuPcnOM34M6CwxW8jH/lxEa5rBoDeUwu5HHTw==",
      "dev": true,
      "bin": {
        "semver": "bin/semver.js"
      }
    },
    "node_modules/@babel/generator": {
      "version": "7.13.9",
      "resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.13.9.tgz",
      "integrity": "sha512-mHOOmY0Axl/JCTkxTU6Lf5sWOg/v8nUa+Xkt4zMTftX0wqmb6Sh7J8gvcehBw7q0AhrhAR+FDacKjCZ2X8K+Sw==",
      "dev": true,
      "dependencies": {
        "@babel/types": "^7.13.0",
        "jsesc": "^2.5.1",
        "source-map": "^0.5.0"
      }
    },
.
.
.
}
```
Note: although both `yarn` and `npm` use `package.json` and `package-lock.json`, it generally advisable to stick to one package manager, rather than switching back and forth between the two.

---

###<a name="vscode"></a>VS Code - (Sorta) Integrated Development Environment 
For all its drawbacks, the [Eclipse IDE] (https://www.eclipse.org/projects/) provides a robust enterprise development environment with a variety of following invaluable features.
{% details Some features that I find invaluable %} 
* code writing with syntax highlighting
* function and method code completion
* easy refactoring of classes, methods, and functions
* synchronization with git version control
* unit testing, coverage, and quality control via integration with, e.g., testng and sonarqube
* robust debugging with tracing and breakpoints
* server management tools for, e.g., Spring Boot, WildFly, JBoss, etc.
 {% enddetails %}

Unfortunately, in our transition to Javascript and AWS-based serverless, we are going to be jettisoning Eclipse and shifting over to a more watered-down development environment: [Visual Studio Code] (https://code.visualstudio.com).  (Note that there are other options out there, such as [Webstorm from JetBrains] (https://www.jetbrains.com/webstorm/), but Visual Studio is arguably the most widely used).

Go ahead and grab a copy from: [https://code.visualstudio.com] (https://code.visualstudio.com).  After installation, you should have a screen like this:

![VS Code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rva9isphougxhgos57k4.png)

We'll get into the details later, but one of the first things you'll notice is that the pulldown menus are not as robust as in Eclipse.  Rather, VS Code has gone the way of `vi`, with a large toolbox of hotkeys and commands hidden from view.  A list of various keyboard shortcuts can be found off the menu:
![VSCode's Keyboard Shortcuts](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2r83dydkrau853ta2hok.png)
as well as [online] (https://code.visualstudio.com/shortcuts/keyboard-shortcuts-macos.pdf).  The primary keystroke to remember is either FN1 or &#8593;&#8984;P to open a command search pulldown that Visual Code calls the "Command Palette" (never mind that this 'palette' is neither a board nor artistic).
![VSCode's So-called Command Palette](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gf31r91nki7coy28kpz4.png)
For now, we are going to want to install the `code` shell command so we can launch VSCode from our working directory in terminal.  So go ahead and type `shell` in the "Command Palette" and select it, which will install the command in your path:
![Shell Command Installation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tw8dwzv8fzk7kz7njiqy.png)
In the bottom right corner, you should receive a popup:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8215jxfrk6pp2r87dqfi.png)
If you pop open terminal (&#8984;-Space then `terminal`), you should now be able to execute the `code` command.

One of VSCode's strengths is a robust selection of extensions, which you can access from the 4 "cubes" icon on the left "Activity Bar". I run with the following basic extensions amongst others:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6ra0yrod1bzre7egnya6.png)
They provide functionality like (a) code cleanup; (b) improved readability; (c) more legible icons in file lists; (d) autocompletion; and (e) code quality checks.

---

###<a name="serverless"></a>Serverless Framework
Next we need to get the tools necessary to start working with Amazon Web Services.  As I noted earlier, I like a formal development environment that includes (a) version control; (b) automated testing; (c) debuggers; and (d) consistent deployment.  Accordingly, I want my deployments done according to a repeatable script that (1) I can check into my version control system; and (2) can be placed into a debug harness.

Unfortunately, AWS historically was poorly designed from this standpoint.  Rather than focus on a command-line driven deployment, application configuration is instead web page driven (an issue now somewhat addressed with the `aws` CLI).

Even now, much of its documentation and tutorials have the user step through web page after web page of deployment, with the underlying configurations created by these pages often hidden from view.  Moreover, it generally lacks the offline lambda function debugger functionality one would want in a true enterprise development environment.

This is where the [Serverless Framework] (https://www.serverless.com/) comes in.  The Serverless Framework provides:

* a level of cloud-server abstraction giving one's code the veneer of avoiding vendor lock-in;
* parameter-based naming and references of resources;
* a robust offline mock server testing environment;
* ability to run attach a debugger to the server; and
* fully scripted deployment of resources

Serverless has both an open-source and a commercial product.  We are only going to cover the open-source solution.  Serverless also provides a web-based monitoring dashboard, but we won't be using this for the time being.

Serverless includes a [relatively decent installation instructions] (https://www.serverless.com/framework/docs/getting-started/) on their website.  Keep in mind, however, that it appears to be a smaller company.  As such, its tutorials seem to be running a few versions behind and are outdated at times with respect to variable names and template structures.

We start off by grabbing and installing the latest version of Serverless (2.37.0 as of April 2021):
```shell
curl -o- -L https://slss.io/install | bash
```
which will result in modifications to our login scripts and installation of `serverless` into `$HOME/.serverless` with output similar to:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/njanbn5yf8rjr7qh8xri.png)

Note we can uninstall (assuming we are not using Docker) by running `serverless uninstall`.  As your current shell likely won't pick up these changes, you'll need to launch a new terminal before continuing.

Switch to your desired base workspace directory.  Go ahead and run `serverless --version`:

```shell
$ serverless --version
Framework Core: 2.37.0 (standalone)
Plugin: 4.5.3
SDK: 4.2.2
Components: 3.8.3
```

##<a name="conclusion"></a>Conclusion
And that's it for the time being.  

Next time, we'll get our AWS account up and running, along with various NPM modules that we will be using for our builds.