PipyJS - A functional style JavaScript engine for programming Pipy
Tags: javascript, opensource, networkproxy, pipy
Published: 2022-03-03 13:00:29


![Pipy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vf68l0zdp8bh6xewgrxl.png)

In previous article [Pipy - A Programmable network proxy for cloud, edge, and IoT](https://dev.to/naqvis/pipy-a-programmable-network-proxy-for-cloud-edge-and-iot-36kb) we introduced [**Pipy**](https://github.com/flomesh-io/pipy), explained its modular design, architecture and crafted a fully functional network proxy with load balancing, caching and routing support. Pipy speaks and understands JavaScript via its custom-developed component **PipyJS**, which is part of the Pipy code base, but has no dependency on it. 

As was seen in Pipy introduction article, all of the network proxy programming logic was written in a JavaScript dialect and this article is going to introduce you to **Pipy**'s powerful programming companion **PipyJS**, the reasoning behind why we chose JavaScript, PipyJS syntax, and areas where PipyJS deviates from ECMAScript. 

## Why JavaScript?

As the name suggests, PipyJS is based off of JavaScript. Pipy has decided to use a JavaScript-like scripting language for a few reasons:

* It's probably [the planet's most widely used programming language](https://insights.stackoverflow.com/survey/2020#technology-programming-scripting-and-markup-languages-professional-developers)
* Its syntax is in C-and-Java-style, which is familiar to the most programmers
* Pipy is a stream processor, that means a large part of its job requires data transformation and restructuring. JavaScript provides a lot of powerful syntax sugar and built-in methods for doing that easily in a very concise way, such as [spread syntax](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Spread_syntax), [destructuring assignment](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment), and Array operations like [map()](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array/map) and [reduce()](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce)
* JSON is the most widely used message format on the web and JSON operations just feel easy and natural in JavaScript
* Last and most importantly, JavaScript embraces the "_functional programming_" paradigm

Pipy works like a bunch of interconnected _pipelines_. Each pipeline is composed of a series of _filters_. Each filter works like a _function_ that gives a certain output in response to its input. For a design pattern like that, a functional programming language would fit Pipy best when your goal is to keep everything coherent and simple.

## Introducing PipyJS

**PipyJS** is a small and embeddable functional-style interpreted JavaScript engine that is designed for high performance with no garbage collection overhead. It supports a subset of ECMAScript standards and in some areas makes bold choices to deviate from them. PipyJS loves and promotes [purely functional](https://en.wikipedia.org/wiki/Functional_programming) style JavaScript, in PipyJS ***everything is an expression***.

### Data types

Like the standard JavaScript, PipyJS supports 4 primitive types and _Object_ type for structural data.

* Primitive types:

  - Undefined. The unique value `undefined` when a variable is not initialized yet.
  - Boolean. `true` and `false`.
  - Number. A 64-bit double precision floating number, such as `123`, `0xabc`, `3.1415926`, `1e6`.
  - String. A sequence of Unicode characters.

* Structural data type:

  - Null object, represented by a unique value `null`.
  - A user defined _plain old data_, such as `{ x: 1.2, y: 300 }`
  - A builtin object, such as _Array_ or _RegExp_.
  - A function, such as `(a, b) => a * a + b * b`.

### Operators

PipyJS supports all standard JavaScript operators, including some operators that were only introduced later in ES2020, such as [optional chaining](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Optional_chaining) and [nullish coalescing](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_operator).

### Global variables

Global variables in PipyJS are also called "_context variables_". The term _context_ here is roughly equivalent to "connection" in terms of high-concurrency network programming, where each connection has their own isolated "_state_" from others. In PipyJS, these isolated states are stored in global variables for convenience. That's why sometimes we call them "_context variables_". That also leads to the fact that PipyJS global variables are different from the standard JavaScript in that they can have different values for different connections. It looks more like [thread-local storage](https://en.wikipedia.org/wiki/Thread-local_storage) in that regard.

Global variables are defined via the built-in function `pipy()`, which is always the very first function you are going to call at the beginning of your script.

``` js
pipy({
  _myGlobalVariable: undefined
})
```

> For convention, we always start **global variable names** with an underscore, though it is not enforced by the language itself.

Global variables are scoped within a single file or `_module_`, and can be shared between different modules by using `export()` and `import()`.

``` js
// file A
pipy().export('namespace-1', {
  __myGlobalVariable: undefined
})

// file B
pipy().import({
  __myGlobalVariable: 'namespace-1'
})
```

> For convention, we always start **exported global variable names** with two underscores, though it is not enforced by the language itself.

### Local variables

In PipyJS, we use function arguments nested inside a function scope for local variables.

``` js
void ((
  x, y, z, // declare local variables as function arguments
) => (
  x = 0,
  y = 0,
  z = 0 // initialize and use the variables in the function body
))() // Don't miss the () to invoke the function right away!
```

> If the above expression is supposed to be evaluated to some return value that will be used afterwards, you should remove the operator `void` at the beginning.

### Branch

In PipyJS, everything is an expression. There is no code blocks or control flow. You can't write _if_ or _for_ statements. But that doesn't mean we can't have branches and loops. We only do it in a different style, the _functional style_.

We can use the logical operator `&&` for simple branches.

``` js
res.status === 200 && (_result = 'OK', console.log('Success.'))

// That's equivalent to:
if (res.status === 200) {
  _result = 'OK';
  console.log('Success.');
}
```

We can combine logical operators `&&` and `||` for multiple-choice branches.

``` js
(res.status === 200) && (
  _result = 'OK'
) ||
(res.status === 404) && (
  _result = 'Not found'
) || (
  _result = ''
)

// That's equivalent to:
if (res.status === 200) {
  _result = 'OK';
} else if (res.status === 404) {
  _result = 'Not found';
} else {
  _result = '';
}
```

### Loop

You can scan an array with [Array.forEach()](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach) for a simple range-loop.

``` js
new Array(100).fill(0).forEach(
  (_, i) => (
    console.log(i)
  )
)

// That's equivalent to:
for (let i = 0; i < 100; i++) {
  console.log(i);
}
```

Or, for a generic conditional loop, you can use the **built-in** function `repeat()`.

``` js
void ((
  n, i
) => (
  n = i = 1,
  repeat(
    () => (
      n *= i,
      i += 1,
      i <= 10
    )
  )
))()

// That's equivalent to:
let n = 1, i = 1;
while (i <= 10) {
  n *= i;
  i += 1;
}
```

## Deviation from ECMAScript

PipyJS is designed specifically for speed and with constructs which are crucial to the design of writing high performant network stream processing logic. Below section highlights the differences where it deviates away from standard ECMAScript or misses the implementation for:

* Object Oriented Programming (OOP) constructs - no user-defined classes or constructors, no _prototype_ system
* Control Flow
* Keywords 
    - `break` , `case` , `catch` , `continue` , `debugger` , `default` , `do` , `else` , `finally` , `function` , `for` , `if` , `return` , `switch` , `throw` , `try` , `while` , `with` , `yield` , `class` , `import` , `export` , `extends` , `static` , `super`
* Type System
  - _BigInt_ and _Symbol_
  - _Strings_ are internally stored as **UTF-8**, and made accessible to scripts as **UTF-32**. For example `"😀".length` is 2 in standard JavaScript, while in PipyJS, it's 1
* Variables - There's no way to declare variables with the `var` or `let` keywords. 

> Though PipyJS deviates from ECMAScript standard, but it does provides **functional style** mechanisms to fill the gaps. 

For API and standard library documentation, please refer to [Website](https://flomesh.io)

## Conclusion

[Pipy](https://github.com/flomesh-io/pipy) is an open-source, extremely fast, and lightweight network traffic processor which can be used in a variety of use cases ranging from edge routers, load balancing & proxying (forward/reverse), API gateways, Static HTTP Servers, Service mesh sidecars, and many other applications. Pipy is in active development and maintained by full-time committers and contributors, though still an early version, it has been battle-tested and in production use by several commercial clients. 

Step-by-step tutorials and documentation can be found on Pipy [GitHub page](https://github.com/flomesh-io/pipy) or accessed via Pipy admin console web UI. The community is welcome to contribute to Pipy development, give it a try for their particular use-case, provide their feedback and insights.

### Related Article
{% embed https://dev.to/naqvis/pipy-a-programmable-network-proxy-for-cloud-edge-and-iot-36kb %}