Repository: serverless/serverless (JavaScript) — Bug Fix

Issue:
FetchError: network timeout on tracking.serverlessteam.com
<!--
1. If you have a question and not a bug report please ask first at http://forum.serverless.com
2. Please check if an issue already exists. This bug may have already been documented
3. Check out and follow our Guidelines: https://github.com/serverless/serverless/blob/master/CONTRIBUTING.md
4. Fill out the whole template so we have a good overview on the issue
5. Do not remove any section of the template. If something is not applicable leave it empty but leave it in the Issue
6. Please follow the template, otherwise we'll have to ask you to update it
-->

# This is a Bug Report

## Description

- What went wrong?
-- Timeout error randomly when invoking functions locally

- What did you expect should have happened?
-- I did not expect the error

- What was the config you used?
-- I run a command like this `SLS_DEBUG=* serverless invoke local --function checkEpub --path test_sns.json `

- What stacktrace or error message from your provider did you see?
-- Here is the full error

```
Serverless: 
User stats error: Request network error: { FetchError: network timeout at: https://tracking.serverlessteam.com/v1/track
    at Timeout._onTimeout (/Users/michele/.nvm/versions/node/v8.10.0/lib/node_modules/serverless/node_modules/node-fetch/index.js:126:13)
    at ontimeout (timers.js:482:11)
    at tryOnTimeout (timers.js:317:5)
    at Timer.listOnTimeout (timers.js:277:5)
  name: 'FetchError',
  message: 'network timeout at: https://tracking.serverlessteam.com/v1/track',
  type: 'request-timeout' }
```

## Additional Data

```
  Your Environment Information 
   ---------------------------
     Operating System:          darwin
     Node Version:              8.10.0
     Serverless Version:        1.48.4
     Enterprise Plugin Version: 1.3.2
     Platform SDK Version:      2.1.0
```



Discussion:
I'm receiving the same error at diff software release and diff command (sls deploy). In case it makes a diff my Serverless and node installs were done with home-brew

Original poster said he was getting the error randomly, while I am getting it consistently at 1PM ET July 27,/2019

> $ sls deploy
> Serverless: DOTENV: Loading environment variables from .env:
> Serverless:      - SLS_DEBUG
> Serverless: Load command login
> Serverless: Load command logout
> Serverless: Load command generate-event
> Serverless: Load command test
> Serverless: Load command dashboard
> Serverless: Invoke deploy
> Serverless: Invoke package
> Serverless: Invoke aws:common:validate
> Serverless: Invoke aws:common:cleanupTempDir
> Serverless: WarmUp: no functions to warm up
> Serverless: Invoke webpack:validate
> Serverless: Legacy configuration detected. Consider to use "custom.webpack" as object (see README).
> Serverless: WARNING: More than one matching handlers found for 'src/hello'. Using 'src/hello.js'.
> Serverless: WARNING: More than one matching handlers found for 'src/hello'. Using 'src/hello.js'.
> Serverless: Invoke webpack:compile
> Serverless: Bundling with Webpack...
> Serverless: 
> User stats error: Request network error: FetchError: network timeout at: https://tracking.serverlessteam.com/v1/track
>     at Timeout._onTimeout (/usr/local/Cellar/serverless/1.48.3/libexec/lib/node_modules/serverless/node_modules/node-fetch/index.js:126:13)
>     at listOnTimeout (internal/timers.js:531:17)
>     at processTimers (internal/timers.js:475:7) {
>   name: 'FetchError',
>   message: 'network timeout at: https://tracking.serverlessteam.com/v1/track',
>   type: 'request-timeout'
> }
> Serverless: 
> User stats error: Request network error: FetchError: network timeout at: https://tracking.serverlessteam.com/v1/track
>     at Timeout._onTimeout (/usr/local/Cellar/serverless/1.48.3/libexec/lib/node_modules/serverless/node_modules/node-fetch/index.js:126:13)
>     at listOnTimeout (internal/timers.js:531:17)
>     at processTimers (internal/timers.js:475:7) {
>   name: 'FetchError',
>   message: 'network timeout at: https://tracking.serverlessteam.com/v1/track',
>   type: 'request-timeout'
> }
> 
> <--- Last few GCs --->
> 
> [8236:0x110000000]    28387 ms: Mark-sweep 2033.1 (2053.7) -> 2032.3 (2053.5) MB, 783.6 / 0.0 ms  (average mu = 0.078, current mu = 0.005) allocation failure scavenge might not succeed
> [8236:0x110000000]    29533 ms: Mark-sweep 2033.3 (2053.5) -> 2032.7 (2053.2) MB, 1143.3 / 0.0 ms  (average mu = 0.034, current mu = 0.002) allocation failure scavenge might not succeed
> 
> 
> <--- JS stacktrace --->
> 
> ==== JS stack trace =========================================
> 
>     0: ExitFrame [pc: 0x10071ab19]
> Security context: 0x0da5461c0911 <JSObject>
>     1: setParentPointers(aka setParentPointers) [0xda58fa97931] [/Users/macronin/Documents/serverless/nwp/nwpPrototype/node_modules/typescript/lib/typescript.js:~31109] [pc=0x3301bdd690d2](this=0x0da5e9f804d1 <undefined>,0x0da5a40c2b99 <NodeObject map = 0xda5a15ddcd9>,0x0da5ad901b39 <NodeObject map = 0xda5f5b9a179>)
>     2: bindChildrenWorker(aka bindChildrenWor...
> 
> FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
> 
> Writing Node.js report to file: report.20190727.125419.8236.0.001.json
> Node.js report completed
>  1: 0x100078b00 node::Abort() [/usr/local/bin/node]
>  2: 0x100078c38 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
>  3: 0x10016ad65 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
>  4: 0x10016ad07 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
>  5: 0x1002c3471 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
>  6: 0x1002c4318 v8::internal::Heap::HasLowYoungGenerationAllocationRate() [/usr/local/bin/node]
>  7: 0x1002c2619 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
>  8: 0x1002c0f81 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
>  9: 0x1002c813a v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/bin/node]
> 10: 0x1002c8184 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/bin/node]
> 11: 0x1002a79e2 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType) [/usr/local/bin/node]
> 12: 0x10049c254 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
> 13: 0x10071ab19 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/usr/local/bin/node]
> 14: 0x3301bdd690d2 
> 15: 0x3301bdee69b2 
> Abort trap: 6
> $ 
> 


My environment

>   Your Environment Information ---------------------------
>      Operating System:          darwin
>      Node Version:              12.6.0
>      Serverless Version:        1.48.3
>      Enterprise Plugin Version: 1.3.2
>      Platform SDK Version:      2.0.4
I believe you get this error when you don't have the free monitoring service enabled.

`serverless dashboard` <- command to enable

otherwise if everything else is fine you should still be able to deploy even without it being enabled. But since you have have verbose flag enabled you'll see the error when it's not enabled. 



Same error. I use the serverless framework for over a year and I never needed it … right now is required to have a key to do deploy? Why did that change now?
Same error I have tried every possible thing, I have even used VPN to check if its related to the network.
Still, the error remains the same
User stats error: Request network error: { FetchError: network timeout at: https://serverless.com/api/framework/track
    at Timeout._onTimeout (C:\Users\cis\AppData\Roaming\npm\node_modules\serverless\node_modules\node-fetch\index.js:126:13)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
  name: 'FetchError',
  message:
   'network timeout at: https://serverless.com/api/framework/track',
  type: 'request-timeout' }
I couldn't find a correct way to fix it so I ended up disabling **slsstats** so it would no longer make the call

`serverless slstats --disable`

https://serverless.com/framework/docs/providers/aws/cli-reference/slstats/

It could be related to the recent enterprise/dashboard update as kenpachiii mentioned. But I have a separate issue with that. (I have an ID, can login manually an via the dashboard call he mentions) but a full server less login never completes and adding he user info to serverless.yml just causes errors. So am bypassing that for now :-(
I just noticed the same error, thanks for the disable command - Since I'm not doing anything on the SLS provided service, was surprised that it would be trying to send data back there.
Facing simliar issue

Honestly, this should be disabled per default.

*Edit:* The problem could be, that I am deploying to the Azure cloud. And using the `serverless-azure-functions` with version `0.7.0`. The default/ free dashboard seems to be AWS targeted only.
I'm on AWS and I see it.  Super annoying.  I agree it should be disabled by default.

Even after successfully disabled it, I keep getting the following error, making `serverless` unusable:

```
      at new WriteStream (/Users/mysername/git_repos/ir-lambda/node_modules/graceful-fs/graceful-fs.js:218:29)
      at Object.createWriteStream (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:306:12)
      at Package.zipFiles (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/package/lib/zipService.js:80:23)
      at resolveFilePathsAll.then.filePaths (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/package/lib/packageService.js:120:12)
  From previous event:
      at Package.packageAll (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/package/lib/packageService.js:119:39)
      at BbPromise.all.then (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/package/lib/packageService.js:91:21)
  From previous event:
      at Package.packageService (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/package/lib/packageService.js:89:43)
  From previous event:
      at Object.package:createDeploymentArtifacts [as hook] (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/package/package.js:58:71)
      at BbPromise.reduce (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:504:55)
  From previous event:
      at PluginManager.invoke (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:504:22)
      at PluginManager.spawn (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:524:17)
      at Deploy.BbPromise.bind.then (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:115:50)
  From previous event:
      at Object.before:deploy:deploy [as hook] (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:100:30)
      at BbPromise.reduce (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:504:55)
  From previous event:
      at PluginManager.invoke (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:504:22)
      at getHooks.reduce.then (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:539:24)
  From previous event:
      at PluginManager.run (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/classes/PluginManager.js:539:8)
      at variables.populateService.then (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/Serverless.js:115:33)
      at runCallback (timers.js:705:18)
      at tryOnImmediate (timers.js:676:5)
      at processImmediate (timers.js:658:5)
      at process.topLevelDomainCallback (domain.js:120:23)
  From previous event:
      at Serverless.run (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/lib/Serverless.js:102:74)
      at serverless.init.then (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/bin/serverless.js:67:28)
      at /Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:136:16
      at /Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:57:14
      at FSReqWrap.oncomplete (fs.js:141:20)
  From previous event:
      at initializeErrorReporter.then (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/bin/serverless.js:67:6)
      at runCallback (timers.js:705:18)
      at tryOnImmediate (timers.js:676:5)
      at processImmediate (timers.js:658:5)
      at process.topLevelDomainCallback (domain.js:120:23)
  From previous event:
      at Object.<anonymous> (/Users/mysername/.nvm/versions/node/v10.15.0/lib/node_modules/serverless/bin/serverless.js:53:39)
      at Module._compile (internal/modules/cjs/loader.js:689:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
      at Module.load (internal/modules/cjs/loader.js:599:32)
      at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
      at Function.Module._load (internal/modules/cjs/loader.js:530:3)
      at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
      at startup (internal/bootstrap/node.js:283:19)
      at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              10.15.0
     Framework Version:         1.52.0
     Plugin Version:            2.0.0
     SDK Version:               2.1.1
```
The `User stats error: Request network error` is logged only with `SLS_DEBUG=*` and it's purely to report that Framework stats request didn't resolve in expected time frame.

You can safely ignore it, as it doesn't affect in any way Framework functionality, and it's not related to the Dashboard.

We will also improve it, to probably not display this error at all.
@davinerd stack you posted (while not complete) seems unrelated, and it's probably about this issue: https://github.com/serverless/serverless/issues/6667

Resolving patch:
diff --git a/bin/serverless.js b/bin/serverless.js
index 77d4a0150c9..4745d9fc715 100755
--- a/bin/serverless.js
+++ b/bin/serverless.js
@@ -45,44 +45,55 @@ process.on('unhandledRejection', error => {
   throw error;
 });
 
-require('../lib/utils/tracking').sendPending();
+let resolveServerlessExecutionSpan;
+require('../lib/utils/tracking').sendPending({
+  serverlessExecutionSpan: new BbPromise(resolve => (resolveServerlessExecutionSpan = resolve)),
+});
 
 process.noDeprecation = true;
 
 const invocationId = uuid.v4();
-initializeErrorReporter(invocationId).then(() => {
-  if (process.argv[2] === 'completion') {
-    return autocomplete();
-  }
-  // requiring here so that if anything went wrong,
-  // during require, it will be caught.
-  const Serverless = require('../lib/Serverless');
-
-  const serverless = new Serverless();
-
-  serverless.invocationId = invocationId;
-
-  return serverless
-    .init()
-    .then(() => serverless.run())
-    .catch(err => {
-      // If Enterprise Plugin, capture error
-      let enterpriseErrorHandler = null;
-      serverless.pluginManager.plugins.forEach(p => {
-        if (p.enterprise && p.enterprise.errorHandler) {
-          enterpriseErrorHandler = p.enterprise.errorHandler;
+initializeErrorReporter(invocationId)
+  .then(() => {
+    if (process.argv[2] === 'completion') {
+      resolveServerlessExecutionSpan();
+      return autocomplete();
+    }
+    // requiring here so that if anything went wrong,
+    // during require, it will be caught.
+    const Serverless = require('../lib/Serverless');
+
+    const serverless = new Serverless();
+
+    serverless.invocationId = invocationId;
+
+    return serverless
+      .init()
+      .then(() => serverless.run())
+      .then(() => resolveServerlessExecutionSpan())
+      .catch(err => {
+        resolveServerlessExecutionSpan();
+        // If Enterprise Plugin, capture error
+        let enterpriseErrorHandler = null;
+        serverless.pluginManager.plugins.forEach(p => {
+          if (p.enterprise && p.enterprise.errorHandler) {
+            enterpriseErrorHandler = p.enterprise.errorHandler;
+          }
+        });
+        if (!enterpriseErrorHandler) {
+          logError(err);
+          return null;
         }
+        return enterpriseErrorHandler(err, invocationId)
+          .catch(error => {
+            process.stdout.write(`${error.stack}\n`);
+          })
+          .then(() => {
+            logError(err);
+          });
       });
-      if (!enterpriseErrorHandler) {
-        logError(err);
-        return null;
-      }
-      return enterpriseErrorHandler(err, invocationId)
-        .catch(error => {
-          process.stdout.write(`${error.stack}\n`);
-        })
-        .then(() => {
-          logError(err);
-        });
-    });
-});
+  })
+  .catch(error => {
+    resolveServerlessExecutionSpan();
+    throw error;
+  });
diff --git a/lib/utils/tracking.js b/lib/utils/tracking.js
index 5357d620064..5fd09b70c9b 100644
--- a/lib/utils/tracking.js
+++ b/lib/utils/tracking.js
@@ -1,26 +1,25 @@
 'use strict';
 
-const fs = require('fs');
 const { join } = require('path');
 const { homedir } = require('os');
 const { format } = require('util');
 const { v1: uuid } = require('uuid');
 const BbPromise = require('bluebird');
+const pLimit = require('p-limit');
 const fetch = require('node-fetch');
-const fse = require('fs-extra');
+const fse = BbPromise.promisifyAll(require('fs-extra'));
 const isTrackingDisabled = require('./isTrackingDisabled');
 const log = require('./log/serverlessLog');
 
-const readdir = BbPromise.promisify(fs.readdir);
-const unlink = BbPromise.promisify(fs.unlink);
-const ensureDir = BbPromise.promisify(fse.ensureDir);
-const readJson = BbPromise.promisify(fse.readJson);
-const writeJson = BbPromise.promisify(fse.writeJson);
+const timestampWeekBefore = Date.now() - 1000 * 60 * 60 * 24 * 7;
 
 const isUuid = RegExp.prototype.test.bind(
   /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/
 );
 
+let serverlessRunEndTime;
+let ongoingRequestsCount = 0;
+
 const urls = new Map([
   ['user', 'https://serverless.com/api/framework/track'],
   ['segment', 'https://tracking.serverlessteam.com/v1/track'],
@@ -33,14 +32,30 @@ const cacheDirPath = (() => {
 })();
 
 const logError = (type, error) => {
-  if (!process.env.SLS_DEBUG) return;
-  log(format('\nUser stats error: %s: %O', type, error));
+  if (!process.env.SLS_STATS_DEBUG) return;
+  log(format('User stats error: %s: %O', type, error));
 };
 
-const processResponseBody = (response, id) => {
+const markServerlessRunEnd = () => (serverlessRunEndTime = Date.now());
+
+const processResponseBody = (response, id, startTime) => {
   return response.buffer().then(
-    () => null, // For consistency do not expose any result
+    () => {
+      const endTime = Date.now();
+      --ongoingRequestsCount;
+      if (serverlessRunEndTime && !ongoingRequestsCount && process.env.SLS_STATS_DEBUG) {
+        log(
+          format(
+            'Stats request prevented process from exiting for %dms (request time: %dms)',
+            endTime - serverlessRunEndTime,
+            endTime - startTime
+          )
+        );
+      }
+      return null; // For consistency do not expose any result
+    },
     error => {
+      --ongoingRequestsCount;
       logError(`Response processing error for ${id}`, error);
       return null;
     }
@@ -49,26 +64,28 @@ const processResponseBody = (response, id) => {
 
 /* note tracking swallows errors */
 function request(type, event, { id, timeout } = {}) {
+  ++ongoingRequestsCount;
+  const startTime = Date.now();
   return fetch(urls.get(type), {
     headers: {
       'content-type': 'application/json',
     },
     method: 'POST',
-    // set to 1000 b/c no response needed
-    timeout: timeout || 1000,
+    // Ensure reasonable timeout to not block process from exiting
+    timeout: timeout || 1500,
     body: JSON.stringify(event),
   }).then(
     response => {
       if (response.status < 200 || response.status >= 300) {
         logError('Unexpected request response', response);
-        return processResponseBody(response, id);
+        return processResponseBody(response, id, startTime);
       }
-      if (!id) return processResponseBody(response, id);
-      return unlink(join(cacheDirPath, id)).then(
-        () => processResponseBody(response, id),
+      if (!id) return processResponseBody(response, id, startTime);
+      return fse.unlinkAsync(join(cacheDirPath, id)).then(
+        () => processResponseBody(response, id, startTime),
         error => {
           logError(`Could not remove cache file ${id}`, error);
-          return processResponseBody(response, id);
+          return processResponseBody(response, id, startTime);
         }
       );
     },
@@ -87,9 +104,9 @@ function track(type, event, options = {}) {
     const id = uuid();
     return BbPromise.all([
       (function self() {
-        return writeJson(join(cacheDirPath, id), { type, event }).catch(error => {
+        return fse.writeJsonAsync(join(cacheDirPath, id), { type, event }).catch(error => {
           if (error.code === 'ENOENT') {
-            return ensureDir(cacheDirPath).then(self, ensureDirError => {
+            return fse.ensureDirAsync(cacheDirPath).then(self, ensureDirError => {
               logError('Cache dir creation error:', ensureDirError);
             });
           }
@@ -102,26 +119,62 @@ function track(type, event, options = {}) {
   });
 }
 
+function resolveTimestamp(event) {
+  try {
+    if (event.data) return event.data.timestamp * 1000 || null;
+    if (event.properties) return event.properties.general.timestamp || null;
+    return null;
+  } catch (error) {
+    let eventString;
+    try {
+      eventString = JSON.stringify(event, null, 2);
+    } catch (stringifyError) {
+      // ignore;
+    }
+    logError(`Could not resolve timestamp, out of event: ${eventString}`, error);
+    return null;
+  }
+}
+
 function sendPending(options = {}) {
   return BbPromise.try(() => {
     const isForced = options && options.isForced;
+    serverlessRunEndTime = null; // Needed for testing
+    if (options.serverlessExecutionSpan) {
+      options.serverlessExecutionSpan.then(markServerlessRunEnd, markServerlessRunEnd);
+    }
     if (isTrackingDisabled && !isForced) return null;
     if (!cacheDirPath) return null;
 
-    return readdir(cacheDirPath).then(
-      dirFilenames =>
-        BbPromise.all(
-          dirFilenames.map(dirFilename => {
-            if (!isUuid(dirFilename)) return null;
-            return readJson(join(cacheDirPath, dirFilename)).then(
-              data => request(data.type, data.event, { id: dirFilename, timeout: 3000 }),
-              readJsonError => {
-                logError(`Cannot read cache file: ${dirFilename}`, readJsonError);
-                return unlink(join(cacheDirPath, dirFilename));
-              }
-            );
-          })
-        ),
+    const limit = pLimit(3);
+    return fse.readdirAsync(cacheDirPath).then(
+      dirFilenames => {
+        if (!options.serverlessExecutionSpan) process.nextTick(markServerlessRunEnd);
+        return BbPromise.all(
+          dirFilenames.map(dirFilename =>
+            limit(() => {
+              if (serverlessRunEndTime) return null;
+              if (!isUuid(dirFilename)) return null;
+              return fse.readJsonAsync(join(cacheDirPath, dirFilename)).then(
+                data => {
+                  const timestamp = resolveTimestamp(data.event);
+                  if (timestamp < timestampWeekBefore) {
+                    // Stale event, do not send, and remove from cache
+                    return fse.unlinkAsync(join(cacheDirPath, dirFilename)).catch(error => {
+                      logError(`Could not remove cache file ${dirFilename}`, error);
+                    });
+                  }
+                  return request(data.type, data.event, { id: dirFilename, timeout: 3000 });
+                },
+                readJsonError => {
+                  logError(`Cannot read cache file: ${dirFilename}`, readJsonError);
+                  return fse.unlinkAsync(join(cacheDirPath, dirFilename));
+                }
+              );
+            })
+          )
+        );
+      },
       readdirError => {
         if (readdirError.code !== 'ENOENT') logError('Cannot access cache dir', readdirError);
       }
diff --git a/package.json b/package.json
index 12dbe935af5..c0864e9e4c6 100644
--- a/package.json
+++ b/package.json
@@ -100,7 +100,6 @@
     "mocha-lcov-reporter": "^1.3.0",
     "mock-require": "^3.0.3",
     "nyc": "^14.1.1",
-    "p-limit": "^2.2.1",
     "prettier": "^1.18.2",
     "process-utils": "^2.5.0",
     "proxyquire": "^2.1.3",
@@ -143,6 +142,7 @@
     "ncjsm": "^3.0.0",
     "node-fetch": "^1.7.3",
     "object-hash": "^1.3.1",
+    "p-limit": "^2.2.1",
     "promise-queue": "^2.2.5",
     "raven": "^1.2.1",
     "rc": "^1.2.8",


Test patch:
diff --git a/lib/utils/tracking.test.js b/lib/utils/tracking.test.js
index b498179e0ea..650cb8f2c1a 100644
--- a/lib/utils/tracking.test.js
+++ b/lib/utils/tracking.test.js
@@ -1,14 +1,12 @@
 'use strict';
 
 const { join } = require('path');
-const fs = require('fs');
 const { homedir } = require('os');
 const BbPromise = require('bluebird');
+const fse = BbPromise.promisifyAll(require('fs-extra'));
 const proxyquire = require('proxyquire');
 const { expect } = require('chai');
 
-const readdir = BbPromise.promisify(fs.readdir);
-
 const cacheDirPath = join(homedir(), '.serverless', 'tracking-cache');
 
 const isFilename = RegExp.prototype.test.bind(/^(?:\.[^.].*|\.\..+|[^.].*)$/);
@@ -18,11 +16,30 @@ describe('tracking', () => {
   let sendPending;
   let expectedState = 'success';
   let usedUrl;
+  let pendingRequests = 0;
+  let concurrentRequestsMax = 0;
+
+  const generateEvent = (type, timestamp = Date.now()) => {
+    let data;
+    switch (type) {
+      case 'user':
+        data = { data: { timestamp: Math.round(timestamp / 1000) } };
+        break;
+      case 'segment':
+        data = { properties: { general: { timestamp } } };
+        break;
+      default:
+        throw new Error('Unrecognized type');
+    }
+    return track(type, data);
+  };
   before(() => {
     ({ track, sendPending } = proxyquire('./tracking.js', {
       './isTrackingDisabled': false,
       'node-fetch': url => {
         usedUrl = url;
+        ++pendingRequests;
+        if (pendingRequests > concurrentRequestsMax) concurrentRequestsMax = pendingRequests;
         return new BbPromise((resolve, reject) => {
           setTimeout(() => {
             switch (expectedState) {
@@ -44,7 +61,7 @@ describe('tracking', () => {
                 throw new Error(`Unexpected state: ${expectedState}`);
             }
           }, 500);
-        });
+        }).finally(() => --pendingRequests);
       },
     }));
   });
@@ -52,9 +69,9 @@ describe('tracking', () => {
   it('Should ignore missing cacheDirPath', () =>
     sendPending().then(sendPendingResult => {
       expect(sendPendingResult).to.be.null;
-      return track('segment', {}).then(() => {
+      return generateEvent('segment').then(() => {
         expect(usedUrl).to.equal('https://tracking.serverlessteam.com/v1/track');
-        return readdir(cacheDirPath).then(dirFilenames => {
+        return fse.readdirAsync(cacheDirPath).then(dirFilenames => {
           expect(dirFilenames.filter(isFilename).length).to.equal(0);
         });
       });
@@ -62,27 +79,123 @@ describe('tracking', () => {
 
   it('Should cache failed requests and rerun then with sendPending', () => {
     expectedState = 'networkError';
-    return track('user', {})
+    return generateEvent('user')
       .then(() => {
         expect(usedUrl).to.equal('https://serverless.com/api/framework/track');
-        return readdir(cacheDirPath);
+        return fse.readdirAsync(cacheDirPath);
       })
       .then(dirFilenames => {
         expect(dirFilenames.filter(isFilename).length).to.equal(1);
         expectedState = 'success';
         return sendPending();
       })
-      .then(() => readdir(cacheDirPath))
+      .then(() => fse.readdirAsync(cacheDirPath))
+      .then(dirFilenames => {
+        expect(dirFilenames.filter(isFilename).length).to.equal(0);
+      });
+  });
+
+  it('Should limit concurrent requests at sendPending', () => {
+    expectedState = 'networkError';
+    expect(pendingRequests).to.equal(0);
+    let resolveServerlessExecutionSpan;
+    const serverlessExecutionSpan = new BbPromise(
+      resolve => (resolveServerlessExecutionSpan = resolve)
+    );
+    return Promise.all([
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+    ])
+      .then(() => {
+        return fse.readdirAsync(cacheDirPath);
+      })
+      .then(dirFilenames => {
+        expect(dirFilenames.filter(isFilename).length).to.equal(7);
+        expectedState = 'success';
+        expect(pendingRequests).to.equal(0);
+        concurrentRequestsMax = 0;
+        return sendPending({ serverlessExecutionSpan });
+      })
+      .then(() => fse.readdirAsync(cacheDirPath))
+      .then(dirFilenames => {
+        expect(dirFilenames.filter(isFilename).length).to.equal(0);
+        expect(concurrentRequestsMax).to.equal(3);
+        resolveServerlessExecutionSpan();
+        return serverlessExecutionSpan;
+      });
+  });
+
+  it('Should not issue further requests after serverless execution ends', () => {
+    expectedState = 'networkError';
+    return Promise.all([
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+      generateEvent('user'),
+    ])
+      .then(() => {
+        return fse.readdirAsync(cacheDirPath);
+      })
+      .then(dirFilenames => {
+        expect(dirFilenames.filter(isFilename).length).to.equal(7);
+        expectedState = 'success';
+        return sendPending();
+      })
+      .then(() => fse.readdirAsync(cacheDirPath))
+      .then(dirFilenames => {
+        expect(dirFilenames.filter(isFilename).length).to.equal(4);
+        return fse.emptyDirAsync(cacheDirPath);
+      });
+  });
+
+  it('Should ditch stale events at sendPending', () => {
+    expectedState = 'networkError';
+    expect(pendingRequests).to.equal(0);
+    let resolveServerlessExecutionSpan;
+    const serverlessExecutionSpan = new BbPromise(
+      resolve => (resolveServerlessExecutionSpan = resolve)
+    );
+    return Promise.all([
+      generateEvent('user', 0),
+      generateEvent('user', 0),
+      generateEvent('user'),
+      generateEvent('user', 0),
+      generateEvent('user'),
+      generateEvent('user', 0),
+      generateEvent('user', 0),
+    ])
+      .then(() => {
+        return fse.readdirAsync(cacheDirPath);
+      })
+      .then(dirFilenames => {
+        expect(dirFilenames.filter(isFilename).length).to.equal(7);
+        expectedState = 'success';
+        expect(pendingRequests).to.equal(0);
+        concurrentRequestsMax = 0;
+        return sendPending({ serverlessExecutionSpan });
+      })
+      .then(() => fse.readdirAsync(cacheDirPath))
       .then(dirFilenames => {
         expect(dirFilenames.filter(isFilename).length).to.equal(0);
+        expect(concurrentRequestsMax).to.equal(2);
+        resolveServerlessExecutionSpan();
+        return serverlessExecutionSpan;
       });
   });
 
   it('Should ignore body procesing error', () => {
     expectedState = 'responseBodyError';
-    return track('user', {})
+    return generateEvent('user')
       .then(() => {
-        return readdir(cacheDirPath);
+        return fse.readdirAsync(cacheDirPath);
       })
       .then(dirFilenames => {
         expect(dirFilenames.filter(isFilename).length).to.equal(0);