Repository: winglang/wing (TypeScript)

Issue:
add queue max retries / retention period
### Community Note

> Please vote by adding a 👍 reaction to the issue to help us prioritize.
> If you are interested to work on this issue, please leave a comment.


### Feature Spec

Add max retries or retention period (or both) to a Queue

### Use Cases

When a queue handler function fails, the message keeps getting pushed back to the queue (infinitely).

We should provide a way to limit the amount of times a function handler is being pushed back to queue.

**Why is it an big issue for us now:**
Running the simulator in the Console with a queue having a faulty handler can cause the Console to crash since the function is being executed repeatedly (and logs are being sent) without the ability to stop it.  

### Implementation Notes

_No response_

### Component

SDK, Wing Console


Discussion: @Chriscbr any comments?
This is an important use case, feels like something we need for beta. 👍 I think max retries is the most straightforward solution. Off the top of my head I'm not sure if this needs to be a configurable property on the queue or on `QueueOnMessageProps` - I'd propose starting with the latter, and implementing it on AWS first to check how it works on their cloud.

> Running the simulator in the Console with a queue having a faulty handler can cause the Console to crash since the function is being executed repeatedly (and logs are being sent) without the ability to stop it.

Is it the number of logs that crashes the console? Or is it some kind of stack overflow within the simulator itself?
Hi,

This issue hasn't seen activity in 60 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days.
Feel free to re-open this issue when there's an update or relevant information to be added.
Thanks!
Fynn is working on it! @staycoolcall911 can you send him an invite - here is his alias: fynnfluegge
I suggest we need to add `retention period` and a `visibility timeout`. I prefer adding default values to both, e.g. 1 day retention and 30s visibility. However, I am not sure what the cause of the simulator crash is right now (smells like a stack overflow of the infinite loop) but the `visibility timeout` will very likely prevent the crash.

Edit: There is already a visibilityTimeout parameter, but it is called `timeout` in `queue` implementation.
Anyway, the `tf-aws` queue is assigning it to `visibilityTimeoutSeconds` from terraform:
![Screenshot 2023-04-10 at 19 45 52](https://user-images.githubusercontent.com/16321871/230960160-88ddb384-8458-4bce-a084-b34ecbe892b4.png)
Seems there could be something wrong with the simulator queue as it defines a default timeout and the simulator still crashes:
![Screenshot 2023-04-10 at 19 50 52](https://user-images.githubusercontent.com/16321871/230961056-c38c4ae9-ba61-4977-83be-d9421d69a803.png)
Guess I will reproduce the simulator crash first 🤠

Suspicious is that the queue simulator expects milliseconds:
![Screenshot 2023-04-10 at 23 09 48](https://user-images.githubusercontent.com/16321871/230999409-ccb762b9-eb25-44df-b1f4-a164e36201d2.png)
but the generator passes seconds, so it's send again every 30ms:
![Screenshot 2023-04-10 at 23 08 45](https://user-images.githubusercontent.com/16321871/230999238-39c85971-a667-4e20-af2a-3f7f3fc0680c.png)





Fix (patch):
diff --git a/docs/04-reference/wingsdk-api.md b/docs/04-reference/wingsdk-api.md
index 42adb3ca736..771bf50a060 100644
--- a/docs/04-reference/wingsdk-api.md
+++ b/docs/04-reference/wingsdk-api.md
@@ -2354,6 +2354,7 @@ let queue_props = cloud.QueueProps{ ... }
 | **Name** | **Type** | **Description** |
 | --- | --- | --- |
 | <code><a href="#@winglang/sdk.cloud.QueueProps.property.initialMessages">initial_messages</a></code> | <code>MutArray&lt;str&gt;</code> | Initialize the queue with a set of messages. |
+| <code><a href="#@winglang/sdk.cloud.QueueProps.property.retentionPeriod">retention_period</a></code> | <code><a href="#@winglang/sdk.std.Duration">Duration</a></code> | How long a queue retains a message. |
 | <code><a href="#@winglang/sdk.cloud.QueueProps.property.timeout">timeout</a></code> | <code><a href="#@winglang/sdk.std.Duration">Duration</a></code> | How long a queue's consumers have to process a message. |
 
 ---
@@ -2371,6 +2372,19 @@ Initialize the queue with a set of messages.
 
 ---
 
+##### `retention_period`<sup>Optional</sup> <a name="retention_period" id="@winglang/sdk.cloud.QueueProps.property.retentionPeriod"></a>
+
+```wing
+retention_period: Duration;
+```
+
+- *Type:* <a href="#@winglang/sdk.std.Duration">Duration</a>
+- *Default:* undefined
+
+How long a queue retains a message.
+
+---
+
 ##### `timeout`<sup>Optional</sup> <a name="timeout" id="@winglang/sdk.cloud.QueueProps.property.timeout"></a>
 
 ```wing
@@ -2378,7 +2392,7 @@ timeout: Duration;
 ```
 
 - *Type:* <a href="#@winglang/sdk.std.Duration">Duration</a>
-- *Default:* Duration.fromSeconds(10)
+- *Default:* undefined
 
 How long a queue's consumers have to process a message.
 
diff --git a/libs/wingsdk/API.md b/libs/wingsdk/API.md
index 8b183b2c528..e5f991a6250 100644
--- a/libs/wingsdk/API.md
+++ b/libs/wingsdk/API.md
@@ -2347,6 +2347,7 @@ let queue_props = cloud.QueueProps{ ... }
 | **Name** | **Type** | **Description** |
 | --- | --- | --- |
 | <code><a href="#@winglang/sdk.cloud.QueueProps.property.initialMessages">initial_messages</a></code> | <code>MutArray&lt;str&gt;</code> | Initialize the queue with a set of messages. |
+| <code><a href="#@winglang/sdk.cloud.QueueProps.property.retentionPeriod">retention_period</a></code> | <code><a href="#@winglang/sdk.std.Duration">Duration</a></code> | How long a queue retains a message. |
 | <code><a href="#@winglang/sdk.cloud.QueueProps.property.timeout">timeout</a></code> | <code><a href="#@winglang/sdk.std.Duration">Duration</a></code> | How long a queue's consumers have to process a message. |
 
 ---
@@ -2364,6 +2365,19 @@ Initialize the queue with a set of messages.
 
 ---
 
+##### `retention_period`<sup>Optional</sup> <a name="retention_period" id="@winglang/sdk.cloud.QueueProps.property.retentionPeriod"></a>
+
+```wing
+retention_period: Duration;
+```
+
+- *Type:* <a href="#@winglang/sdk.std.Duration">Duration</a>
+- *Default:* undefined
+
+How long a queue retains a message.
+
+---
+
 ##### `timeout`<sup>Optional</sup> <a name="timeout" id="@winglang/sdk.cloud.QueueProps.property.timeout"></a>
 
 ```wing
@@ -2371,7 +2385,7 @@ timeout: Duration;
 ```
 
 - *Type:* <a href="#@winglang/sdk.std.Duration">Duration</a>
-- *Default:* Duration.fromSeconds(10)
+- *Default:* undefined
 
 How long a queue's consumers have to process a message.
 
diff --git a/libs/wingsdk/src/cloud/queue.ts b/libs/wingsdk/src/cloud/queue.ts
index 8bb60527c66..bc4139ff7b3 100644
--- a/libs/wingsdk/src/cloud/queue.ts
+++ b/libs/wingsdk/src/cloud/queue.ts
@@ -15,10 +15,16 @@ export const QUEUE_FQN = fqnForType("cloud.Queue");
 export interface QueueProps {
   /**
    * How long a queue's consumers have to process a message.
-   * @default Duration.fromSeconds(10)
+   * @default undefined
    */
   readonly timeout?: Duration;
 
+  /**
+   * How long a queue retains a message.
+   * @default undefined
+   */
+  readonly retentionPeriod?: Duration;
+
   /**
    * Initialize the queue with a set of messages.
    * @default []
diff --git a/libs/wingsdk/src/target-awscdk/queue.ts b/libs/wingsdk/src/target-awscdk/queue.ts
index adc343bdfff..41cc673b646 100644
--- a/libs/wingsdk/src/target-awscdk/queue.ts
+++ b/libs/wingsdk/src/target-awscdk/queue.ts
@@ -25,6 +25,9 @@ export class Queue extends cloud.Queue {
       visibilityTimeout: props.timeout
         ? Duration.seconds(props.timeout?.seconds)
         : undefined,
+      retentionPeriod: props.retentionPeriod
+        ? Duration.seconds(props.retentionPeriod?.seconds)
+        : undefined,
     });
 
     if ((props.initialMessages ?? []).length) {
diff --git a/libs/wingsdk/src/target-sim/queue.inflight.ts b/libs/wingsdk/src/target-sim/queue.inflight.ts
index 41a14f6faaa..cf257bc9f38 100644
--- a/libs/wingsdk/src/target-sim/queue.inflight.ts
+++ b/libs/wingsdk/src/target-sim/queue.inflight.ts
@@ -16,18 +16,24 @@ import {
 export class Queue
   implements IQueueClient, ISimulatorResourceInstance, IEventPublisher
 {
-  private readonly messages = new Array<string>();
+  private readonly messages = new Array<QueueMessage>();
   private readonly subscribers = new Array<QueueSubscriber>();
   private readonly intervalId: NodeJS.Timeout;
   private readonly context: ISimulatorContext;
   private readonly timeout: number;
+  private readonly retentionPeriod: number;
 
   constructor(props: QueueSchema["props"], context: ISimulatorContext) {
     if (props.initialMessages) {
-      this.messages.push(...props.initialMessages);
+      this.messages.push(
+        ...props.initialMessages.map(
+          (message) => new QueueMessage(this.retentionPeriod, message)
+        )
+      );
     }
 
     this.timeout = props.timeout;
+    this.retentionPeriod = props.retentionPeriod;
     this.intervalId = setInterval(() => this.processMessages(), 100); // every 0.1 seconds
     this.context = context;
   }
@@ -56,7 +62,7 @@ export class Queue
     return this.context.withTrace({
       message: `Push (message=${message}).`,
       activity: async () => {
-        this.messages.push(message);
+        this.messages.push(new QueueMessage(this.retentionPeriod, message));
       },
     });
   }
@@ -87,7 +93,8 @@ export class Queue
       // assumptions on the order that subscribers process messages.
       for (const subscriber of new RandomArrayIterator(this.subscribers)) {
         const messages = this.messages.splice(0, subscriber.batchSize);
-        if (messages.length === 0) {
+        const messagesPayload = messages.map((m) => m.payload);
+        if (messagesPayload.length === 0) {
           continue;
         }
         const fnClient = this.context.findInstance(
@@ -100,47 +107,55 @@ export class Queue
           type: TraceType.RESOURCE,
           data: {
             message: `Sending messages (messages=${JSON.stringify(
-              messages
+              messagesPayload
             )}, subscriber=${subscriber.functionHandle}).`,
           },
           sourcePath: this.context.resourcePath,
           sourceType: QUEUE_TYPE,
           timestamp: new Date().toISOString(),
         });
-        void fnClient.invoke(JSON.stringify({ messages })).catch((err) => {
-          // If the function returns an error, put the message back on the queue after timeout period
-          this.context.addTrace({
-            data: {
-              message: `Subscriber error - returning ${messages.length} messages to queue: ${err.message}`,
-            },
-            sourcePath: this.context.resourcePath,
-            sourceType: QUEUE_TYPE,
-            type: TraceType.RESOURCE,
-            timestamp: new Date().toISOString(),
-          });
-          void this.pushMessagesBackToQueue(messages).catch((requeueErr) => {
+        void fnClient
+          .invoke(JSON.stringify({ messages: messagesPayload }))
+          .catch((err) => {
+            // If the function returns an error, put the message back on the queue after timeout period
             this.context.addTrace({
               data: {
-                message: `Error pushing ${messages.length} messages back to queue: ${requeueErr.message}`,
+                message: `Subscriber error - returning ${messagesPayload.length} messages to queue: ${err.message}`,
               },
               sourcePath: this.context.resourcePath,
               sourceType: QUEUE_TYPE,
               type: TraceType.RESOURCE,
               timestamp: new Date().toISOString(),
             });
+            void this.pushMessagesBackToQueue(messages).catch((requeueErr) => {
+              this.context.addTrace({
+                data: {
+                  message: `Error pushing ${messagesPayload.length} messages back to queue: ${requeueErr.message}`,
+                },
+                sourcePath: this.context.resourcePath,
+                sourceType: QUEUE_TYPE,
+                type: TraceType.RESOURCE,
+                timestamp: new Date().toISOString(),
+              });
+            });
           });
-        });
         processedMessages = true;
       }
     } while (processedMessages);
   }
 
-  public async pushMessagesBackToQueue(messages: Array<string>): Promise<void> {
+  public async pushMessagesBackToQueue(
+    messages: Array<QueueMessage>
+  ): Promise<void> {
     setTimeout(() => {
-      this.messages.push(...messages);
+      // Don't push back messages with retention timeouts that have expired
+      const retainedMessages = messages.filter(
+        (message) => message.retentionTimeout > new Date()
+      );
+      this.messages.push(...retainedMessages);
       this.context.addTrace({
         data: {
-          message: `${messages.length} messages pushed back to queue after timeout.`,
+          message: `${retainedMessages.length} messages pushed back to queue after visibility timeout.`,
         },
         sourcePath: this.context.resourcePath,
         sourceType: QUEUE_TYPE,
@@ -151,6 +166,18 @@ export class Queue
   }
 }
 
+class QueueMessage {
+  retentionTimeout: Date;
+  payload: string;
+
+  constructor(retentionPeriod: number, message: string) {
+    const currentTime = new Date();
+    currentTime.setSeconds(retentionPeriod + currentTime.getSeconds());
+    this.retentionTimeout = currentTime;
+    this.payload = message;
+  }
+}
+
 class RandomArrayIterator<T = any> implements Iterable<T> {
   private length: number;
   constructor(private readonly values: T[]) {
diff --git a/libs/wingsdk/src/target-sim/queue.ts b/libs/wingsdk/src/target-sim/queue.ts
index a4839284e62..5c9995d57b9 100644
--- a/libs/wingsdk/src/target-sim/queue.ts
+++ b/libs/wingsdk/src/target-sim/queue.ts
@@ -18,11 +18,20 @@ import { convertBetweenHandlers } from "../utils/convert";
  */
 export class Queue extends cloud.Queue implements ISimulatorResource {
   private readonly timeout: Duration;
+  private readonly retentionPeriod: Duration;
   private readonly initialMessages: string[] = [];
   constructor(scope: Construct, id: string, props: cloud.QueueProps = {}) {
     super(scope, id, props);
 
     this.timeout = props.timeout ?? Duration.fromSeconds(10);
+    this.retentionPeriod = props.retentionPeriod ?? Duration.fromHours(1);
+
+    if (this.retentionPeriod < this.timeout) {
+      throw new Error(
+        "Retention period must be greater than or equal to timeout"
+      );
+    }
+
     this.initialMessages.push(...(props.initialMessages ?? []));
   }
 
@@ -92,6 +101,7 @@ export class Queue extends cloud.Queue implements ISimulatorResource {
       path: this.node.path,
       props: {
         timeout: this.timeout.seconds,
+        retentionPeriod: this.retentionPeriod.seconds,
         initialMessages: this.initialMessages,
       },
       attrs: {} as any,
diff --git a/libs/wingsdk/src/target-sim/schema-resources.ts b/libs/wingsdk/src/target-sim/schema-resources.ts
index 3ad76a6b19d..6a792cb68a5 100644
--- a/libs/wingsdk/src/target-sim/schema-resources.ts
+++ b/libs/wingsdk/src/target-sim/schema-resources.ts
@@ -71,6 +71,8 @@ export interface QueueSchema extends BaseResourceSchema {
   readonly props: {
     /** How long a queue's consumers have to process a message, in seconds */
     readonly timeout: number;
+    /** How long a queue retains a message, in seconds */
+    readonly retentionPeriod: number;
     /** Initial messages to be pushed to the queue. */
     readonly initialMessages: string[];
   };
diff --git a/libs/wingsdk/src/target-tf-aws/queue.ts b/libs/wingsdk/src/target-tf-aws/queue.ts
index abd224dc156..91a7080344c 100644
--- a/libs/wingsdk/src/target-tf-aws/queue.ts
+++ b/libs/wingsdk/src/target-tf-aws/queue.ts
@@ -32,6 +32,7 @@ export class Queue extends cloud.Queue {
 
     this.queue = new SqsQueue(this, "Default", {
       visibilityTimeoutSeconds: props.timeout?.seconds,
+      messageRetentionSeconds: props.retentionPeriod?.seconds,
       name: ResourceNames.generateName(this, NAME_OPTS),
     });