Repository: awslabs/aws-lambda-rust-runtime (language: rust)

Issue:
Lambda events and the optionality of attributes - issues to clarify
Hi there!

Thanks for the massive amount of work put into developing this project. I've got an idea for an improvement I'd like to share.

Looking at the implementation of models for lambda events, some doubts have emerged in several places, which may require clarification and possibly correction.

Can I ask about the source on which the models for lambda events are implemented? I've scoured the entire internet, looking not only for written documentation but also Smithy, etc. At most, I found EventBridge Schemas, but not for lambda events. 
I sporadically managed to find something in AWS Documentation, but only for a few events.

And where I did find a contract definition, I discovered inconsistencies in the implementation in this repository.

Let's take a closer look at the following:

- https://docs.aws.amazon.com/kinesis/latest/APIReference/API_Record.html versus https://github.com/awslabs/aws-lambda-rust-runtime/blob/main/lambda-events/src/event/kinesis/event.rs
  - `partition_key` and `sequence_number` are not optional (described as "Required: Yes" in API Reference)
  - `approximate_arrival_timestamp` shall be optional
  - the `encryption_type` might be defined as enum (NONE, KMS) instead of String

I could probably send a Pull Request, but it will involve breaking changes. I'm not sure if this is something that has to wait for a Major Release. On the other hand, these seem to be important issues.

And this raises the question of how we can verify the remaining events? That's why I was asking about the specification on which these things were implemented.




Discussion / hints:
> I could probably send a Pull Request, but it will involve breaking changes. I'm not sure if this is something that has to wait for a Major Release. On the other hand, these seem to be important issues.

We make these kind of changes all the time. We're not guaranteeing backwards compatibility between minor versions at the moment. See my latest PR where I changed an optional string to an enum: https://github.com/awslabs/aws-lambda-rust-runtime/pull/856/files#diff-68127a0a666444f6d22fb233e1b76b2be5f9379e390679c4e9cdcba5d1ce0a9cR24


> And this raises the question of how we can verify the remaining events? That's why I was asking about the specification on which these things were implemented.

Unfortunately, there is no easy way to verify events except by triggering them and inspecting the payloads. All these events were generated based on the [Go events](https://github.com/aws/aws-lambda-go/tree/3a93ed1e265b5e368f0d20c3a2b8395d2c9f787d/events). Go doesn't have great string semantics, so everything that was a `string` in Go was translated as `Option<String>` in Rust.

We've improved many events with the help of contributors like yourself, so don't hesitate to send PRs that can benefit other people..


Fix patch:
diff --git a/lambda-events/src/event/kinesis/event.rs b/lambda-events/src/event/kinesis/event.rs
index 2b14cbed..94ebbbf9 100644
--- a/lambda-events/src/event/kinesis/event.rs
+++ b/lambda-events/src/event/kinesis/event.rs
@@ -62,15 +62,24 @@ pub struct KinesisEventRecord {
 pub struct KinesisRecord {
     pub approximate_arrival_timestamp: SecondTimestamp,
     pub data: Base64Data,
-    pub encryption_type: Option<String>,
     #[serde(default)]
-    pub partition_key: Option<String>,
+    pub encryption_type: KinesisEncryptionType,
     #[serde(default)]
-    pub sequence_number: Option<String>,
+    pub partition_key: String,
+    #[serde(default)]
+    pub sequence_number: String,
     #[serde(default)]
     pub kinesis_schema_version: Option<String>,
 }
 
+#[derive(Clone, Debug, Default, Deserialize, Eq, PartialEq, Serialize)]
+#[serde(rename_all = "SCREAMING_SNAKE_CASE")]
+pub enum KinesisEncryptionType {
+    #[default]
+    None,
+    Kms,
+}
+
 #[cfg(test)]
 mod test {
     use super::*;
@@ -80,6 +89,20 @@ mod test {
     fn example_kinesis_event() {
         let data = include_bytes!("../../fixtures/example-kinesis-event.json");
         let parsed: KinesisEvent = serde_json::from_slice(data).unwrap();
+        assert_eq!(KinesisEncryptionType::None, parsed.records[0].kinesis.encryption_type);
+
+        let output: String = serde_json::to_string(&parsed).unwrap();
+        let reparsed: KinesisEvent = serde_json::from_slice(output.as_bytes()).unwrap();
+        assert_eq!(parsed, reparsed);
+    }
+
+    #[test]
+    #[cfg(feature = "kinesis")]
+    fn example_kinesis_event_encrypted() {
+        let data = include_bytes!("../../fixtures/example-kinesis-event-encrypted.json");
+        let parsed: KinesisEvent = serde_json::from_slice(data).unwrap();
+        assert_eq!(KinesisEncryptionType::Kms, parsed.records[0].kinesis.encryption_type);
+
         let output: String = serde_json::to_string(&parsed).unwrap();
         let reparsed: KinesisEvent = serde_json::from_slice(output.as_bytes()).unwrap();
         assert_eq!(parsed, reparsed);
diff --git a/lambda-events/src/fixtures/example-kinesis-event-encrypted.json b/lambda-events/src/fixtures/example-kinesis-event-encrypted.json
new file mode 100644
index 00000000..7d1697b1
--- /dev/null
+++ b/lambda-events/src/fixtures/example-kinesis-event-encrypted.json
@@ -0,0 +1,37 @@
+{
+	"Records": [
+		{
+			"kinesis": {
+				"kinesisSchemaVersion": "1.0",
+				"partitionKey": "s1",
+				"sequenceNumber": "49568167373333333333333333333333333333333333333333333333",
+				"data": "SGVsbG8gV29ybGQ=",
+				"approximateArrivalTimestamp": 1480641523.477,
+				"encryptionType": "KMS"
+			},
+			"eventSource": "aws:kinesis",
+			"eventVersion": "1.0",
+			"eventID": "shardId-000000000000:49568167373333333333333333333333333333333333333333333333",
+			"eventName": "aws:kinesis:record",
+			"invokeIdentityArn": "arn:aws:iam::123456789012:role/LambdaRole",
+			"awsRegion": "us-east-1",
+			"eventSourceARN": "arn:aws:kinesis:us-east-1:123456789012:stream/simple-stream"
+		},
+		{
+			"kinesis": {
+				"kinesisSchemaVersion": "1.0",
+				"partitionKey": "s1",
+				"sequenceNumber": "49568167373333333334444444444444444444444444444444444444",
+				"data": "SGVsbG8gV29ybGQ=",
+				"approximateArrivalTimestamp": 1480841523.477
+			},
+			"eventSource": "aws:kinesis",
+			"eventVersion": "1.0",
+			"eventID": "shardId-000000000000:49568167373333333334444444444444444444444444444444444444",
+			"eventName": "aws:kinesis:record",
+			"invokeIdentityArn": "arn:aws:iam::123456789012:role/LambdaRole",
+			"awsRegion": "us-east-1",
+			"eventSourceARN": "arn:aws:kinesis:us-east-1:123456789012:stream/simple-stream"
+		}
+	]
+}
\ No newline at end of file


Test patch: