Repository: pinojs/pino
Pull request #708 (2019-09-11 23:40:38)

Add MySQL/MariaDB transport
Hi - I've added a transport for MySQL/MariaDB which can extract fields from anywhere in the log stream and map them to database columns. It can also save the entire stream as a JSON field in MySQL. I found this useful for my latest project using the wonderful pino logger.
You can check the release here: https://www.npmjs.com/package/pino-mysql


Patch:
diff --git a/README.md b/README.md
index 1452aad..04a4070 100644
--- a/README.md
+++ b/README.md
@@ -51,6 +51,7 @@ For using Pino with a web framework see:
 * [Pino with Restify](docs/web.md#restify)
 * [Pino with Koa](docs/web.md#koa)
 * [Pino with Node core `http`](docs/web.md#http)
+* [Pino with Nest](docs/web.md#nest)
 
 
 <a name="essentials"></a>
diff --git a/docs/api.md b/docs/api.md
index 400eb3a..7e7afae 100644
--- a/docs/api.md
+++ b/docs/api.md
@@ -434,6 +434,13 @@ Write a `'error'` level log, if the configured `level` allows for it.
 
 Write a `'fatal'` level log, if the configured `level` allows for it.
 
+Since `'fatal'` level messages are intended to be logged just prior to the process exiting the `fatal`
+method will always sync flush the destination.
+Therefore it's important not to misuse `fatal` since
+it will cause performance overhead if used for any
+other purpose than writing final log messages before
+the process crashes or exits.
+
 * See [`mergingObject` log method parameter](#mergingobject)
 * See [`message` log method parameter](#message)
 * See [`...interpolationValues` log method parameter](#interpolationvalues)
@@ -508,7 +515,7 @@ const child = logger.child({ foo: 'bar' })
 console.log(child.bindings())
 // { foo: 'bar' }
 const anotherChild = child.child({ MIX: { IN: 'always' } })
-console.log(child.bindings())
+console.log(anotherChild.bindings())
 // { foo: 'bar', MIX: { IN: 'always' } }
 ```
 
diff --git a/docs/transports.md b/docs/transports.md
index 56c31b0..9b33723 100644
--- a/docs/transports.md
+++ b/docs/transports.md
@@ -75,6 +75,7 @@ PR's to this document are welcome for any new transports!
 + [pino-datadog](#pino-datadog)
 + [pino-elasticsearch](#pino-elasticsearch)
 + [pino-mq](#pino-mq)
++ [pino-mysql](#pino-mysql)
 + [pino-papertrail](#pino-papertrail)
 + [pino-redis](#pino-redis)
 + [pino-sentry](#pino-sentry)
@@ -163,7 +164,7 @@ To connect to an external elasticsearch instance (recommended for production):
 * Launch:
 
 ```sh
-$ node app.js | pino-elasticsearch --host 192.168.1.42
+$ node app.js | pino-elasticsearch --node http://192.168.1.42:9200
 ```
 
 Assuming Elasticsearch is running on `192.168.1.42`.
@@ -171,7 +172,7 @@ Assuming Elasticsearch is running on `192.168.1.42`.
 To connect to AWS Elasticsearch:
 
 ```sh
-$ node app.js | pino-elasticsearch  --host https://es-url.us-east-1.es.amazonaws.com --port 443 -c ./aws_config.json
+$ node app.js | pino-elasticsearch --node https://es-url.us-east-1.es.amazonaws.com --es-version 6
 ```
 
 Then [create an index pattern](https://www.elastic.co/guide/en/kibana/current/setup.html) on `'pino'` (the default index key for `pino-elasticsearch`) on the Kibana instance.
@@ -222,6 +223,25 @@ node yourapp.js | pino-papertrail --host bar.papertrailapp.com --port 12345 --ap
 
 for full documentation of command line switches read [readme](https://github.com/ovhemert/pino-papertrail#readme)
 
+<a id="pino-mysql"></a>
+### pino-mysql
+
+[pino-mysql][pino-mysql] loads pino logs into [MySQL][MySQL] and [MariaDB][MariaDB].
+
+```sh
+$ node app.js | pino-mysql -c db-configuration.json
+```
+
+`pino-mysql` can extract and save log fields into corresponding database field
+and/or save the entire log stream as a [JSON Data Type][JSONDT]. 
+
+For full documentation and command line switches read the [readme][pino-mysql].
+
+[pino-mysql]: https://www.npmjs.com/package/pino-mysql
+[MySQL]: https://www.mysql.com/
+[MariaDB]: https://mariadb.org/
+[JSONDT]: https://dev.mysql.com/doc/refman/8.0/en/json.html
+
 <a id="pino-redis"></a>
 ### pino-redis
 
diff --git a/docs/web.md b/docs/web.md
index d12c76b..8871a48 100644
--- a/docs/web.md
+++ b/docs/web.md
@@ -9,12 +9,13 @@ web framework ecosystem.
 + [Pino with Restify](#restify)
 + [Pino with Koa](#koa)
 + [Pino with Node core `http`](#http)
++ [Pino with Nest](#nest)
 
 <a id="fastify"></a>
 ## Pino with Fastify
 
 The Fastify web framework comes bundled with Pino by default, simply set Fastify's
-`logger` option to `true` and use `reply.log` for log messages that correspond
+`logger` option to `true` and use `request.log` or `reply.log` for log messages that correspond
 to each individual request:
 
 ```js
@@ -188,3 +189,42 @@ server.listen(3000)
 ```
 
 See the [pino-http readme](http://npm.im/pino-http) for more info.
+
+
+<a id="nest"></a>
+## Pino with Nest
+
+```sh
+npm install nestjs-pino
+```
+
+```ts
+import { NestFactory } from '@nestjs/core'
+import { Controller, Get, Module } from '@nestjs/common'
+import { LoggerModule, Logger } from 'nestjs-pino'
+
+@Controller()
+export class AppController {
+  constructor(private readonly logger: Logger) {}
+
+  @Get()
+  getHello() {
+    this.logger.log('something')
+    return `Hello world`
+  }
+}
+
+@Module({
+  controllers: [AppController],
+  imports: [LoggerModule.forRoot()]
+})
+class MyModule {}
+
+async function bootstrap() {
+  const app = await NestFactory.create(MyModule)
+  await app.listen(3000)
+}
+bootstrap()
+```
+
+See the [nestjs-pino readme](http://npm.im/nestjs-pino) for more info.
diff --git a/lib/levels.js b/lib/levels.js
index ea8c33b..a9e0e55 100644
--- a/lib/levels.js
+++ b/lib/levels.js
@@ -1,6 +1,13 @@
 'use strict'
 const flatstr = require('flatstr')
-const { lsCacheSym, levelValSym, useLevelLabelsSym, changeLevelNameSym, useOnlyCustomLevelsSym } = require('./symbols')
+const {
+  lsCacheSym,
+  levelValSym,
+  useLevelLabelsSym,
+  changeLevelNameSym,
+  useOnlyCustomLevelsSym,
+  streamSym
+} = require('./symbols')
 const { noop, genLog } = require('./tools')
 
 const levels = {
@@ -11,9 +18,15 @@ const levels = {
   error: 50,
   fatal: 60
 }
-
+const logFatal = genLog(levels.fatal)
 const levelMethods = {
-  fatal: genLog(levels.fatal),
+  fatal (...args) {
+    const stream = this[streamSym]
+    logFatal.call(this, ...args)
+    if (typeof stream.flushSync === 'function') {
+      stream.flushSync()
+    }
+  },
   error: genLog(levels.error),
   warn: genLog(levels.warn),
   info: genLog(levels.info),
diff --git a/lib/proto.js b/lib/proto.js
index 1ec71cf..09cc0d9 100644
--- a/lib/proto.js
+++ b/lib/proto.js
@@ -72,12 +72,24 @@ function child (bindings) {
   const instance = Object.create(this)
   if (bindings.hasOwnProperty('serializers') === true) {
     instance[serializersSym] = Object.create(null)
+
     for (var k in serializers) {
       instance[serializersSym][k] = serializers[k]
     }
+    const parentSymbols = Object.getOwnPropertySymbols(serializers)
+    for (var i = 0; i < parentSymbols.length; i++) {
+      const ks = parentSymbols[i]
+      instance[serializersSym][ks] = serializers[ks]
+    }
+
     for (var bk in bindings.serializers) {
       instance[serializersSym][bk] = bindings.serializers[bk]
     }
+    const bindingsSymbols = Object.getOwnPropertySymbols(bindings.serializers)
+    for (var bi = 0; bi < bindingsSymbols.length; bi++) {
+      const bks = bindingsSymbols[bi]
+      instance[serializersSym][bks] = bindings.serializers[bks]
+    }
   } else instance[serializersSym] = serializers
   if (bindings.hasOwnProperty('customLevels') === true) {
     assertNoLevelCollisions(this.levels, bindings.customLevels)
@@ -95,8 +107,8 @@ function bindings () {
   const chindings = this[chindingsSym]
   var chindingsJson = `{${chindings.substr(1)}}` // at least contains ,"pid":7068,"hostname":"myMac"
   var bindingsFromJson = JSON.parse(chindingsJson)
-  delete bindingsFromJson['pid']
-  delete bindingsFromJson['hostname']
+  delete bindingsFromJson.pid
+  delete bindingsFromJson.hostname
   return bindingsFromJson
 }
 
@@ -125,7 +137,7 @@ function write (_obj, msg, num) {
     }
   }
 
-  const s = this[asJsonSym](obj, num, t, objError)
+  const s = this[asJsonSym](obj, num, t)
 
   const stream = this[streamSym]
   if (stream[needsMetadataGsym] === true) {
diff --git a/lib/redaction.js b/lib/redaction.js
index 5e7fd92..c1dd721 100644
--- a/lib/redaction.js
+++ b/lib/redaction.js
@@ -17,30 +17,33 @@ function redaction (opts, serialize) {
 
   const shape = paths.reduce((o, str) => {
     rx.lastIndex = 0
-    rx.exec(str)
+    const first = rx.exec(str)
     const next = rx.exec(str)
+
+    // ns is the top-level path segment, brackets + quoting removed.
+    let ns = first[1] !== undefined
+      ? first[1].replace(/^(?:"|'|`)(.*)(?:"|'|`)$/, '$1')
+      : first[0]
+
+    if (ns === '*') {
+      ns = wildcardFirstSym
+    }
+
     // top level key:
     if (next === null) {
-      if (str === '*') {
-        str = wildcardFirstSym
-      }
-      o[str] = null
+      o[ns] = null
       return o
     }
-    const { index } = next
-    const firstPosBracket = str[index - 1] === '['
-    const leadingChar = firstPosBracket ? '[' : ''
-    const nextPath = `${leadingChar}${str.substr(index, str.length - 1)}`
-    var ns = str.substr(0, index - 1).replace(/^\["(.+)"\]$/, '$1')
-    if (ns === '*') {
-      ns = wildcardFirstSym
-    }
 
+    // path with at least two segments:
     // if ns is already redacted at the top level, ignore lower level redactions
     if (o[ns] === null) {
       return o
     }
 
+    const { index } = next
+    const nextPath = `${str.substr(index, str.length - 1)}`
+
     o[ns] = o[ns] || []
 
     // shape is a mix of paths beginning with literal values and wildcard
diff --git a/package.json b/package.json
index 63fdd98..93f1262 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "pino",
-  "version": "5.13.2",
+  "version": "5.13.5",
   "description": "super fast, all natural json logger",
   "main": "pino.js",
   "browser": "./browser.js",
@@ -62,22 +62,22 @@
     "benchmark": "^2.1.4",
     "bole": "^3.0.2",
     "bunyan": "^1.8.12",
-    "cross-env": "^5.2.0",
+    "cross-env": "^5.2.1",
     "docsify-cli": "^4.2.1",
     "execa": "^1.0.0",
     "fastbench": "^1.0.1",
     "flush-write-stream": "^2.0.0",
     "import-fresh": "^3.0.0",
     "log": "^5.0.0",
-    "loglevel": "^1.6.1",
+    "loglevel": "^1.6.4",
     "pino-pretty": "^2.4.0",
     "pre-commit": "^1.2.2",
-    "proxyquire": "^2.1.0",
+    "proxyquire": "^2.1.3",
     "pump": "^3.0.0",
     "qodaa": "^1.0.1",
     "snazzy": "^8.0.0",
     "split2": "^3.1.1",
-    "standard": "^13.1.0",
+    "standard": "^14.2.0",
     "steed": "^1.1.3",
     "tap": "^12.6.0",
     "tape": "^4.10.1",
@@ -85,11 +85,11 @@
     "winston": "^3.2.1"
   },
   "dependencies": {
-    "fast-redact": "^1.4.4",
-    "fast-safe-stringify": "^2.0.6",
+    "fast-redact": "^2.0.0",
+    "fast-safe-stringify": "^2.0.7",
     "flatstr": "^1.0.9",
     "pino-std-serializers": "^2.3.0",
-    "quick-format-unescaped": "^3.0.2",
+    "quick-format-unescaped": "^3.0.3",
     "sonic-boom": "^0.7.5"
   }
 }


Test patch:
diff --git a/test/basic.test.js b/test/basic.test.js
index e4ff500..a83c6e5 100644
--- a/test/basic.test.js
+++ b/test/basic.test.js
@@ -107,6 +107,19 @@ function levelTest (name, level) {
     check(is, result, level, 'hello 42')
   })
 
+  test(`formatting a symbol at level ${name}`, async ({ is }) => {
+    const stream = sink()
+    const instance = pino(stream)
+    instance.level = name
+
+    const sym = Symbol('foo')
+    instance[name]('hello', sym)
+
+    const result = await once(stream, 'data')
+
+    check(is, result, level, 'hello Symbol(foo)')
+  })
+
   test(`passing error with a serializer at level ${name}`, async ({ is, same }) => {
     const stream = sink()
     const err = new Error('myerror')
diff --git a/test/browser-serializers.test.js b/test/browser-serializers.test.js
index 5ff787a..6ed78d5 100644
--- a/test/browser-serializers.test.js
+++ b/test/browser-serializers.test.js
@@ -30,7 +30,8 @@ test('serializers override values', ({ end, is }) => {
 })
 
 test('without the serialize option, serializers do not override values', ({ end, is }) => {
-  const parent = pino({ serializers: parentSerializers,
+  const parent = pino({
+    serializers: parentSerializers,
     browser: {
       write (o) {
         is(o.test, 'test')
@@ -164,12 +165,14 @@ if (process.title !== 'browser') {
 
     console.error = consoleError
 
-    logger.child({ aBinding: 'test',
+    logger.child({
+      aBinding: 'test',
       serializers: {
         key: () => 'serialized',
         key2: () => 'serialized2',
         key3: () => 'serialized3'
-      } }).fatal({ key: 'test' }, { key2: 'test' }, 'str should skip', [{ foo: 'array should skip' }], { key3: 'test' })
+      }
+    }).fatal({ key: 'test' }, { key2: 'test' }, 'str should skip', [{ foo: 'array should skip' }], { key3: 'test' })
     end()
   })
 
@@ -204,17 +207,20 @@ if (process.title !== 'browser') {
 
     console.error = consoleError
 
-    logger.child({ key: 'test',
+    logger.child({
+      key: 'test',
       serializers: {
         key: () => 'serialized'
-      } }).fatal({ test: 'test' })
+      }
+    }).fatal({ test: 'test' })
     end()
   })
 }
 
 test('child does not overwrite parent serializers', ({ end, is }) => {
   var c = 0
-  const parent = pino({ serializers: parentSerializers,
+  const parent = pino({
+    serializers: parentSerializers,
     browser: {
       serialize: true,
       write (o) {
@@ -224,7 +230,9 @@ test('child does not overwrite parent serializers', ({ end, is }) => {
           is(o.test, 'child')
           end()
         }
-      } } })
+      }
+    }
+  })
   const child = parent.child({ serializers: childSerializers })
 
   parent.fatal({ test: 'test' })
@@ -232,7 +240,8 @@ test('child does not overwrite parent serializers', ({ end, is }) => {
 })
 
 test('children inherit parent serializers', ({ end, is }) => {
-  const parent = pino({ serializers: parentSerializers,
+  const parent = pino({
+    serializers: parentSerializers,
     browser: {
       serialize: true,
       write (o) {
diff --git a/test/custom-levels.test.js b/test/custom-levels.test.js
index 7eec50f..283504d 100644
--- a/test/custom-levels.test.js
+++ b/test/custom-levels.test.js
@@ -210,9 +210,11 @@ test('throws when useOnlyCustomLevels is set true without customLevels', async (
 })
 
 test('custom level on one instance does not affect other instances', async ({ is }) => {
-  pino({ customLevels: {
-    foo: 37
-  } })
+  pino({
+    customLevels: {
+      foo: 37
+    }
+  })
   is(typeof pino().foo, 'undefined')
 })
 
diff --git a/test/fixtures/pretty/custom-time-label.js b/test/fixtures/pretty/custom-time-label.js
index 161b691..4384034 100644
--- a/test/fixtures/pretty/custom-time-label.js
+++ b/test/fixtures/pretty/custom-time-label.js
@@ -3,7 +3,7 @@ Date.now = function () { return 1459875739796 }
 require('os').hostname = function () { return 'abcdefghijklmnopqr' }
 var pino = require(require.resolve('./../../../'))
 var log = pino({
-  timestamp: () => `,"custom-time-label":"test"`,
+  timestamp: () => ',"custom-time-label":"test"',
   prettyPrint: true
 })
 log.info('h')
diff --git a/test/fixtures/pretty/custom-time.js b/test/fixtures/pretty/custom-time.js
index f93d726..092632b 100644
--- a/test/fixtures/pretty/custom-time.js
+++ b/test/fixtures/pretty/custom-time.js
@@ -3,7 +3,7 @@ Date.now = function () { return 1459875739796 }
 require('os').hostname = function () { return 'abcdefghijklmnopqr' }
 var pino = require(require.resolve('./../../../'))
 var log = pino({
-  timestamp: () => `,"time":"test"`,
+  timestamp: () => ',"time":"test"',
   prettyPrint: true
 })
 log.info('h')
diff --git a/test/levels.test.js b/test/levels.test.js
index eacb744..4643ad8 100644
--- a/test/levels.test.js
+++ b/test/levels.test.js
@@ -393,7 +393,7 @@ test('throws when creating a default value that does not exist in logger levels'
       useOnlyCustomLevels: true
     })
   } catch ({ message }) {
-    is(message, `default level:info must be included in custom levels`)
+    is(message, 'default level:info must be included in custom levels')
   }
 })
 
@@ -402,3 +402,18 @@ test('passes when creating a default value that exists in logger levels', async
     level: 30
   })
 })
+
+test('fatal method sync-flushes the destination if sync flushing is available', async ({ pass, doesNotThrow, plan }) => {
+  plan(2)
+  const stream = sink()
+  stream.flushSync = () => {
+    pass('destination flushed')
+  }
+  const instance = pino(stream)
+  instance.fatal('this is fatal')
+  await once(stream, 'data')
+  doesNotThrow(() => {
+    stream.flushSync = undefined
+    instance.fatal('this is fatal')
+  })
+})
diff --git a/test/pretty.test.js b/test/pretty.test.js
index 976f37f..caf9f62 100644
--- a/test/pretty.test.js
+++ b/test/pretty.test.js
@@ -62,7 +62,7 @@ test('throws when prettyPrint is true but pino-pretty module is not installed',
   // error is to simulate it:
   const prettyFactory = require('pino-pretty')
   require.cache[require.resolve('pino-pretty')].exports = () => {
-    throw Error(`Cannot find module 'pino-pretty'`)
+    throw Error('Cannot find module \'pino-pretty\'')
   }
   throws(() => pino({ prettyPrint: true }))
   try { pino({ prettyPrint: true }) } catch ({ message }) {
diff --git a/test/redact.test.js b/test/redact.test.js
index 26db3ac..5a812cc 100644
--- a/test/redact.test.js
+++ b/test/redact.test.js
@@ -606,22 +606,88 @@ test('redacts null at the top level', async ({ is }) => {
 
 test('supports bracket notation', async ({ is }) => {
   const stream = sink()
-  const instance = pino({ redact: ['a["b-b"]'] }, stream)
+  const instance = pino({ redact: ['a["b.b"]'] }, stream)
   const obj = {
-    a: { 'b-b': 'c' }
+    a: { 'b.b': 'c' }
   }
   instance.info(obj)
   const o = await once(stream, 'data')
-  is(o.a['b-b'], '[Redacted]')
+  is(o.a['b.b'], '[Redacted]')
 })
 
-test('supports leading bracket notation', async ({ is }) => {
+test('supports bracket notation with further nesting', async ({ is }) => {
   const stream = sink()
-  const instance = pino({ redact: ['["a-a"].b'] }, stream)
+  const instance = pino({ redact: ['a["b.b"].c'] }, stream)
   const obj = {
-    'a-a': { b: 'c' }
+    a: { 'b.b': { c: 'd' } }
   }
   instance.info(obj)
   const o = await once(stream, 'data')
-  is(o['a-a'].b, '[Redacted]')
+  is(o.a['b.b'].c, '[Redacted]')
+})
+
+test('supports bracket notation with empty string as path segment', async ({ is }) => {
+  const stream = sink()
+  const instance = pino({ redact: ['a[""].c'] }, stream)
+  const obj = {
+    a: { '': { c: 'd' } }
+  }
+  instance.info(obj)
+  const o = await once(stream, 'data')
+  is(o.a[''].c, '[Redacted]')
+})
+
+test('supports leading bracket notation (single quote)', async ({ is }) => {
+  const stream = sink()
+  const instance = pino({ redact: ['[\'a.a\'].b'] }, stream)
+  const obj = {
+    'a.a': { b: 'c' }
+  }
+  instance.info(obj)
+  const o = await once(stream, 'data')
+  is(o['a.a'].b, '[Redacted]')
+})
+
+test('supports leading bracket notation (double quote)', async ({ is }) => {
+  const stream = sink()
+  const instance = pino({ redact: ['["a.a"].b'] }, stream)
+  const obj = {
+    'a.a': { b: 'c' }
+  }
+  instance.info(obj)
+  const o = await once(stream, 'data')
+  is(o['a.a'].b, '[Redacted]')
+})
+
+test('supports leading bracket notation (backtick quote)', async ({ is }) => {
+  const stream = sink()
+  const instance = pino({ redact: ['[`a.a`].b'] }, stream)
+  const obj = {
+    'a.a': { b: 'c' }
+  }
+  instance.info(obj)
+  const o = await once(stream, 'data')
+  is(o['a.a'].b, '[Redacted]')
+})
+
+test('supports leading bracket notation (single-segment path)', async ({ is }) => {
+  const stream = sink()
+  const instance = pino({ redact: ['[`a.a`]'] }, stream)
+  const obj = {
+    'a.a': { b: 'c' }
+  }
+  instance.info(obj)
+  const o = await once(stream, 'data')
+  is(o['a.a'], '[Redacted]')
+})
+
+test('supports leading bracket notation (single-segment path, wilcard)', async ({ is }) => {
+  const stream = sink()
+  const instance = pino({ redact: ['[*]'] }, stream)
+  const obj = {
+    'a.a': { b: 'c' }
+  }
+  instance.info(obj)
+  const o = await once(stream, 'data')
+  is(o['a.a'], '[Redacted]')
 })
diff --git a/test/serializers.test.js b/test/serializers.test.js
index c6a7079..70e8a6a 100644
--- a/test/serializers.test.js
+++ b/test/serializers.test.js
@@ -124,6 +124,32 @@ test('children inherit parent serializers', async ({ is }) => {
   is(o.test, 'parent')
 })
 
+test('children inherit parent Symbol serializers', async ({ is, isNot }) => {
+  const stream = sink()
+  const symbolSerializers = {
+    [Symbol.for('pino.*')]: parentSerializers.test
+  }
+  const parent = pino({ serializers: symbolSerializers }, stream)
+
+  is(parent[Symbol.for('pino.serializers')], symbolSerializers)
+
+  const child = parent.child({
+    serializers: {
+      [Symbol.for('a')]: a,
+      a
+    }
+  })
+
+  function a () {
+    return 'hello'
+  }
+
+  isNot(child[Symbol.for('pino.serializers')], symbolSerializers)
+  is(child[Symbol.for('pino.serializers')].a, a)
+  is(child[Symbol.for('pino.serializers')][Symbol.for('a')], a)
+  is(child[Symbol.for('pino.serializers')][Symbol.for('pino.*')], parentSerializers.test)
+})
+
 test('children serializers get called', async ({ is }) => {
   const stream = sink()
   const parent = pino({